HUL715 Paper Replication

Forecasting the Yield Spread Using PCA, Vasicek and ARIMA Models

Abstract of the paper

In this research paper, the author has applied various econometric time series and two machine learning models (MLPs, LSTMs) to forecast the daily data on the yield spread i.e. the difference between the 10-year Treasury yields and the 3-month Treasury bills.

First, we decomposed the yield curve into its principal components, then simulated various paths of the yield spread using the Vasicek model. After constructing univariate ARIMA models, and multivariate models such as ARIMAX, VAR and Long Short Term Memory (LSTM), I calibrated the root mean squared error (RMSE) to measure how far the models’ results deviate from the current values. Through impulse response functions, we measure the impact of various shocks on the difference yield spread.

The results indicate that the parsimonious univariate ARIMA model (RMSE = 0.05185) outperforms the richly parameterized VAR method (RMSE = 0.4648) , and the complex LSTM with multivariate data performs equally well as the simple ARIMA model.

Gathering data and environment setup

In [ ]:
!pip install arch
!pip install fredapi
!pip install quandl
In [ ]:
import warnings
warnings.filterwarnings('ignore')

import math
from math import sqrt
import datetime
from datetime import date, timedelta
import numpy as np
import pandas as pd
from pandas import Series
from pandas import DataFrame
import scipy
import scipy.stats as scs
import statsmodels.api as sm

from fbprophet import Prophet # for forecasting via ML called "Prophet"
from fbprophet.plot import plot_plotly
import logging
logging.getLogger().setLevel(logging.ERROR) # mutes unimportant diagnostic messages



import sklearn.decomposition as sck_dec # has PCA functionalities
import sklearn.decomposition.pca as PCA
from sklearn.decomposition import PCA
from sklearn.preprocessing import StandardScaler # for standardizing the Data
from sklearn.preprocessing import LabelEncoder
from sklearn.metrics import mean_squared_error
from statsmodels.tsa.arima_model import ARIMA
from arch import arch_model # has ARCH and GARCH models

import seaborn as sn

import matplotlib
from matplotlib import pyplot
import matplotlib.pyplot as plt
import matplotlib.patches as mpatches
# matplotlib.style.use('dark_background')
matplotlib.style.use('default')
%matplotlib inline
import matplotlib.pyplot as plt


# standardizes the size of all plots, rather than typing plot(figsize = (12,6)) everytime
from pylab import rcParams
rcParams['figure.figsize'] = 12,6 

import seaborn as sns
from plotly import __version__ 
import cufflinks as cf 
from IPython.display import display, HTML, Image
import plotly
from plotly.offline import download_plotlyjs, init_notebook_mode, plot, iplot
import plotly.graph_objects as go
import plotly.figure_factory as ff
import plotly.express as px

# set plotly to offline mode for easy reproducibilty
import plotly.offline as py
py.init_notebook_mode()

from statsmodels.tsa.stattools import adfuller,grangercausalitytests
from statsmodels.tsa.stattools import ccovf,ccf,periodogram
from statsmodels.graphics.tsaplots import plot_acf,plot_pacf


# Deep learning packages
import keras
from keras.models import Sequential
from keras.layers import LSTM, Dense
from keras.layers import Dropout
from keras.layers import *
from sklearn.preprocessing import MinMaxScaler
from sklearn.metrics import mean_squared_error
from sklearn.metrics import mean_absolute_error
from keras.callbacks import EarlyStopping

import warnings
warnings.filterwarnings('ignore')
In [ ]:
from fredapi import Fred
import quandl

quandl.ApiConfig.api_key = '-kesKc4A8jkkBwyX3rSX'
fred = Fred(api_key='08ac10f3d6d00a973c9cdec45990f255')  
In [ ]:
today_date = "2020-12-15"
paper_end_date = "2020-08-21"

#edit end_date to control the end_date of the API call
end_date = today_date
In [ ]:
def parser(x):
    return datetime.strptime('190'+x, '%Y-%m')

treas = ['FRED/DGS3MO',
           'FRED/DGS6MO',
           'FRED/DGS1',
           'FRED/DGS2',
           'FRED/DGS3',
           'FRED/DGS5',
           'FRED/DGS7',
           'FRED/DGS10',
           'FRED/DGS20',
           'FRED/DGS30',
           'FRED/T10Y3M'] # 10-Year Treasury Constant Maturity Minus 3-Month Treasury Constant Maturity

data = quandl.get(treas, start_date = "1993-10-01", end_date=end_date,
                  order='asc', collapse = 'daily', date_parser=parser)
data.columns = ['tres3mo', 
                    'tres6mo',
                    'tres1y',
                    'tres2y',
                    'tres3y',
                    'tres5y',
                    'tres7y',
                    'tres10y',
                    'tres20y',
                    'tres30y',
                    'yieldsp']
In [ ]:
print(data.shape)
data
(6806, 11)
Out[ ]:
tres3mo tres6mo tres1y tres2y tres3y tres5y tres7y tres10y tres20y tres30y yieldsp
Date
1993-10-01 2.98 3.11 3.35 3.84 4.18 4.72 5.03 5.34 6.12 5.98 2.36
1993-10-04 3.02 3.17 3.35 3.85 4.16 4.71 5.04 5.34 6.10 5.99 2.32
1993-10-05 3.06 3.20 3.38 3.86 4.18 4.72 5.05 5.35 6.12 6.01 2.29
1993-10-06 3.04 3.19 3.36 3.85 4.16 4.70 5.05 5.35 6.12 6.01 2.31
1993-10-07 3.05 3.17 3.35 3.83 4.15 4.69 5.05 5.33 6.11 6.01 2.28
... ... ... ... ... ... ... ... ... ... ... ...
2020-12-09 0.08 0.09 0.10 0.16 0.21 0.41 0.68 0.95 1.48 1.69 0.87
2020-12-10 0.08 0.09 0.10 0.14 0.20 0.39 0.65 0.92 1.44 1.65 0.84
2020-12-11 0.08 0.08 0.10 0.11 0.18 0.37 0.63 0.90 1.42 1.63 0.82
2020-12-14 0.09 0.08 0.10 0.13 0.17 0.37 0.63 0.90 1.43 1.63 0.81
2020-12-15 0.08 0.08 0.09 0.11 0.18 0.37 0.64 0.92 1.45 1.65 0.84

6806 rows × 11 columns

In [ ]:
data.describe()
Out[ ]:
tres3mo tres6mo tres1y tres2y tres3y tres5y tres7y tres10y tres20y tres30y yieldsp
count 6806.000000 6806.000000 6806.000000 6806.000000 6806.000000 6806.000000 6806.000000 6806.000000 6806.000000 6806.000000 6806.000000
mean 2.375043 2.503572 2.623221 2.877467 3.073964 3.439993 3.741892 3.979077 4.493748 4.545438 1.604020
std 2.138845 2.181814 2.184475 2.176942 2.107344 1.955291 1.848460 1.722358 1.684365 1.526726 1.119141
min 0.000000 0.020000 0.080000 0.110000 0.100000 0.190000 0.360000 0.520000 0.870000 0.990000 -0.950000
25% 0.160000 0.240000 0.410000 0.790000 1.110000 1.690000 2.130000 2.470000 2.890000 3.100000 0.660000
50% 1.740000 1.900000 2.100000 2.440000 2.600000 3.090000 3.550000 3.990000 4.640000 4.640000 1.570000
75% 4.780000 4.980000 4.960000 4.910000 4.910000 5.020000 5.220000 5.270000 5.820000 5.670000 2.550000
max 6.420000 6.670000 7.320000 7.740000 7.840000 7.900000 7.920000 8.050000 8.300000 8.160000 3.850000
In [ ]:
# check if data contains nan
np.isnan(data).any() 
Out[ ]:
tres3mo    False
tres6mo    False
tres1y     False
tres2y     False
tres3y     False
tres5y     False
tres7y     False
tres10y    False
tres20y    False
tres30y    False
yieldsp    False
dtype: bool
In [ ]:
#fill nan values using the fill forward heuristic in case there are any nan values

data_df = data.interpolate(method='spline', order=3).replace([np.inf, -np.inf], 0)
#data_df = data.fillna(method='ffill').replace([np.inf, -np.inf], 0)
np.isnan(data_df).any() 
Out[ ]:
tres3mo    False
tres6mo    False
tres1y     False
tres2y     False
tres3y     False
tres5y     False
tres7y     False
tres10y    False
tres20y    False
tres30y    False
yieldsp    False
dtype: bool
In [ ]:
#data1 is just the data DataFrame without the yeildspread variable
data1 = data_df.loc[:, data.columns != 'yieldsp']
data1.shape
Out[ ]:
(6806, 10)

10Y minus 3M treasury i.e. Yield Spread

In [ ]:
# 10-Year Treasury Constant Maturity Minus 3-Month Treasury Constant Maturity
def parser(x):
    return datetime.strptime('190'+x, '%Y-%m')

yieldsp = quandl.get('FRED/T10Y3M', start_date = "1982-01-04", end_date= end_date,collapse = 'daily',
                  date_parser=parser)
yieldsp.columns = ['yieldsp']
yieldsp.tail()
Out[ ]:
yieldsp
Date
2020-12-09 0.87
2020-12-10 0.84
2020-12-11 0.82
2020-12-14 0.81
2020-12-15 0.84
In [ ]:
yieldsp.shape
Out[ ]:
(9741, 1)
In [ ]:
yieldsp.describe()
Out[ ]:
yieldsp
count 9741.000000
mean 1.744351
std 1.122410
min -0.960000
25% 0.830000
50% 1.800000
75% 2.650000
max 5.180000

Data visualisation

Plotting treasury yield rates for different maturities

In [ ]:
#plotly has some compatibility issues with colab, just taking care of those
import plotly.io as pio
pio.renderers.default = 'notebook'
In [ ]:
tresury_rates_plot = data1.iplot(asFigure=True, kind='scatter', xTitle='Year', yTitle='Treasury Rates')

tresury_rates_plot.update_layout( title={ 'text': "Treasury Yields of Different Maturities",
                                         'y':0.9, 'x':0.5, 'xanchor': 'center', 'yanchor': 'top'})
tresury_rates_plot.layout.template = 'seaborn'
tresury_rates_plot.show()

Plot an interactive 3D yield curve for varying maturities

In [ ]:
ids = ['3-month', '6-month', '1-year','2-year', '5-year', '7-year', '10-year', '20-year', '30-year']

plotly.offline.iplot({
    "data": [
        go.Surface(
            x=ids,
            y=data1.index,
            z=data1.values,
            # colorscale='Greens',

            # reverse scale and remove colorbar
            reversescale=True,
            showscale=True,

            # show contours on x axis only
            contours={
                'x': {
                    'highlight': True,
                    'highlightcolor': 'white',
                    'highlightwidth': 2
                },
                'y': {
                    'highlight': True,
                    'highlightcolor': 'white',
                    'highlightwidth': 3
                },
                'z': {
                    'highlight': False
                }
            })
    ],

    "layout":
    go.Layout(
        title='Interactive 3D Yield Curve',
        width=1200,
        height=800,

        # change aspect ratio and axis titles
        scene={
            'aspectmode': 'manual',
            'aspectratio': {
                'x': 5,
                'y': 15,
                'z': 3
            },
            'yaxis': {
                'title': ''
            },
            'xaxis': {
                'title': 'Maturity duration',
                'tickmode': 'linear',
                'tickfont': {'size': 10}
            },
            'zaxis': {
                'title': 'Treasury Yield (in percent)',
                'tickmode': 'linear',
                'tickfont': {'size': 10},
                'showticklabels': True,
                'showgrid': True
            }
        },
        hoverlabel={'bgcolor': 'turquoise'})
})

Plot the yield spread from Jan, 2005

In [ ]:
# Using graph_objects

yield_df = go.Scatter(x = yieldsp.index, y = yieldsp['yieldsp'], line=dict(color='green', width= 1))

layout = go.Layout(title='Yield Spread (10 Year minus 3 Month Treasury Rate) between Jan 1, 2005 and Dec 15, 2020', 
                   title_font_size=15, 
                   xaxis_title="", yaxis_title="Yield Spread (in percentages)", 
                   xaxis_range=[datetime.datetime(2005, 1, 1),
                   datetime.datetime(2020, 12, 15)])

fig = go.Figure(data=[yield_df], layout=layout)
iplot(fig)

Methods and Results

Principal Component Analysis (PCA)

PCA extracts the signal and diminishes the dataset's dimensionality. This is because it finds the fewest number of variables that explain the largest proportion of the data.

The author has decomposed the data into 5 principal components and has then calculated the proportion of total variance explained by each of the components.

In [ ]:
X = data1.values 

# Standard Scaler transforms dataframe into a normal distribution
sc = StandardScaler().fit_transform(X)
# replace the nan values in the sc array to 0
sc[np.isnan(sc)] = 0 
In [ ]:
# do PCA with n_components left
pca = PCA(n_components = 5)
pcs = pca.fit_transform(sc)

pc_df = pd.DataFrame(data = pcs, columns = ['PC1', 'PC2', 'PC3', 'PC4', 'PC5'])

pc_df['PC2'] = pc_df['PC2']*(-1)   # the negative of PC2 is highly correlated to the yield spread 
pc_yieldsp = pd.merge(pc_df, data[['tres10y', 'yieldsp']], 
                      on = data.index).rename(columns= {'key_0': 'Date'}).set_index('Date')
pc_yieldsp.head()
Out[ ]:
PC1 PC2 PC3 PC4 PC5 tres10y yieldsp
Date
1993-10-01 1.862628 0.786127 0.021733 0.011433 0.038445 5.34 2.36
1993-10-04 1.873816 0.767228 0.040964 0.004595 0.026150 5.34 2.32
1993-10-05 1.905672 0.762668 0.053980 0.005000 0.025452 5.35 2.29
1993-10-06 1.890554 0.772518 0.056664 0.004707 0.027388 5.35 2.31
1993-10-07 1.876005 0.771827 0.061369 0.000007 0.020266 5.33 2.28

Plotting the principal components

In [ ]:
fig_pc = pc_yieldsp[['PC1', 'PC2', 'PC3']].iplot(asFigure=True, 
                      kind='scatter', xTitle='Year', yTitle='Principal Component')
fig_pc.update_layout(
    title={
        'text': "Three Principal Components (PC) that Constitute the Yield Curve",
        'y':0.9,
        'x':0.5,
        'xanchor': 'center',
        'yanchor': 'top'})

fig_pc.show()

Exploring the proportion of the total variance explained by each component

We can calculate the proportion of the total variance explained by each $PC_i$ by the relation:

$$V (PC_i) = \frac{\lambda_i}{\lambda_1+\lambda_2+...+\lambda_5}$$
In [ ]:
# returns Percentage of variance explained by each of the selected components

ratio = pca.explained_variance_ratio_               
ratio_df = pd.DataFrame(ratio, columns=['Percentage of variance explained by component'])  
print(ratio_df)
   Percentage of variance explained by component
0                                       0.935798
1                                       0.059747
2                                       0.003486
3                                       0.000454
4                                       0.000241

Scree plot.): Plotting the proportion of variance explained by each component

In [ ]:
fig_ratio = ratio_df.iplot(asFigure=True, kind='scatter', #layout = layout,
                                            xTitle='Principal Component', yTitle='Eigenvalue (Proportion)')
fig_ratio.update_layout(
    title={
        'text': "Scree Plot: Proportion of Total Variance Explained by each PC",
        'y':0.9,
        'x':0.5,
        'xanchor': 'center',
        'yanchor': 'top'})

fig_ratio.show()

The plot of the first principal component looks very similar to the actual 10-year yield curve. This is consistent with our expectation as the first principal component explains 93.52% of the data.

In [ ]:
fig_pc1 = pc_yieldsp[['PC1','tres10y']].iplot(asFigure=True, 
                      kind='scatter', xTitle='Year', yTitle='Rates')
fig_pc1.update_layout(
    title={
        'text': "PC1 and 10-year Treasury Yield",
        'y':0.9,
        'x':0.5,
        'xanchor': 'center',
        'yanchor': 'top'})

fig_pc1.show()

The second principal component depicts the slope, which in this case is the difference between the 10 year treasury bond and the 3 month tresury bill, also called the yield spread. Visually, the slope appears nearly identical to $PC_2$.

In [ ]:
fig_pc2 = pc_yieldsp[['PC2','yieldsp']].iplot(asFigure=True, 
                      kind='scatter', xTitle='Year', yTitle='Rates')

fig_pc2.update_layout(
    title={
        'text': "PC2 and Yield Spread (10y-3m Treasury Rate)",
        'y':0.9,
        'x':0.5,
        'xanchor': 'center',
        'yanchor': 'top'})

fig_pc2.show()

The high correlation of 0.916 between the 10Y-3M spread and $PC_2$ corroborates the evidence that the second principal component denotes the slope.

Correlation matrix

In [ ]:
#Correlation matrix 

pc_yieldsp[['PC1', 'PC2', 'PC3','tres10y', 'yieldsp']].corr()
Out[ ]:
PC1 PC2 PC3 tres10y yieldsp
PC1 1.000000e+00 4.117721e-16 -1.768370e-16 0.974053 -0.311746
PC2 4.117721e-16 1.000000e+00 -8.436025e-16 0.223748 0.927806
PC3 -1.768370e-16 -8.436025e-16 1.000000e+00 -0.012598 -0.188014
tres10y 9.740531e-01 2.237479e-01 -1.259788e-02 1.000000 -0.092277
yieldsp -3.117462e-01 9.278059e-01 -1.880136e-01 -0.092277 1.000000
In [ ]:
f = plt.figure(figsize=(8, 6))
plt.matshow(pc_yieldsp[['PC1', 'PC2', 'PC3','tres10y', 'yieldsp']].corr(), fignum=f.number)
plt.xticks(range(pc_yieldsp[['PC1', 'PC2', 'PC3','tres10y', 'yieldsp']].shape[1]), pc_yieldsp[['PC1', 'PC2', 'PC3','tres10y', 'yieldsp']].columns, fontsize=14, rotation=45)
plt.yticks(range(pc_yieldsp[['PC1', 'PC2', 'PC3','tres10y', 'yieldsp']].shape[1]), pc_yieldsp[['PC1', 'PC2', 'PC3','tres10y', 'yieldsp']].columns, fontsize=14)
cb = plt.colorbar()
cb.ax.tick_params(labelsize=14)
plt.title('Correlation Matrix', fontsize=16);

Vasicek Interest Rate Model

  • A stochastic technique of modeling the instantaneous movements in the term structure of forward interest rates is the Vasicek interest rate model.
  • The factors that describe the trajectory of interest rates are time, market risk, and equilibrium value, wherein the model assumes that the rates revert to the mean of those factors as time passes.

This method employs maximum likelihood estimation to derive the parameters of the Vasicek model, which is of the form:

$$ dr_t = k(\theta-r_t) dt + \sigma dW_t $$

where, $k=$ strength of mean reversion or the speed at which the yield spread rates revert to the mean $\theta$;

$\theta$ is the level of mean reversion or the long-run level of yield spread

$\sigma$ is the volatility in yield spread at time t

$r_t $ is the short rate (yield spread) at time t

$k(\theta - r_t)$ are the expected changes in the yield spread or the drift term, also known as the mean reversion for Vasicek model, and

$W$ is the random market risk that follows a Wiener process.

Using the closed form solution given below that avoids “discretization errors”, we have simulated the paths of the yield spread:

$$ r_{t_i} = r_{t_{i-1}} e^{-k(t_i-t_{i-1})} + \theta (1- e^{-k(t_i-t_{i-1})}) + Z \sqrt{\frac{\sigma^2 (1- e^{-2k(t_i-t_{i-1})} )}{2k}} $$

where $Z \sim N(0,1)$

In [ ]:
# helper functions for Vasicek interest rate model 

def VasicekNextRate(r, kappa, theta, sigma, dt=1/252):
    # Implements above closed form solution     
    val1 = np.exp(-1*kappa*dt)
    val2 = (sigma**2)*(1-val1**2) / (2*kappa)
    out = r*val1 + theta*(1-val1) + (np.sqrt(val2))*np.random.normal()
    return out



def VasicekSim(N, r0, kappa, theta, sigma, dt = 1/252):
    short_r = [0]*N # Create array to store rates     short_r[0] = r0 # Initialise rates at $r_0$     
    
    for i in range(1,N):
        short_r[i] = VasicekNextRate(short_r[i-1], kappa, theta, sigma, dt)
    
    return short_r




def VasicekMultiSim(M, N, r0, kappa, theta, sigma, dt = 1/252):
    sim_arr = np.ndarray((N, M))
    
    for i in range(0,M):
        sim_arr[:, i] = VasicekSim(N, r0, kappa, theta, sigma, dt)
    
    return sim_arr



def VasicekCalibration(rates, dt=1/252):
    n = len(rates)
    
    # Implement MLE to calibrate parameters 
    Sx = sum(rates[0:(n-1)])
    Sy = sum(rates[1:n])
    Sxx = np.dot(rates[0:(n-1)], rates[0:(n-1)])
    Sxy = np.dot(rates[0:(n-1)], rates[1:n])
    Syy = np.dot(rates[1:n], rates[1:n])
    
    theta = (Sy * Sxx - Sx * Sxy) / (n * (Sxx - Sxy) - (Sx**2 - Sx*Sy))
    kappa = -np.log((Sxy - theta * Sx - theta * Sy + n * theta**2) / (Sxx - 2*theta*Sx + n*theta**2)) / dt
    a = np.exp(-kappa * dt)
    sigmah2 = (Syy - 2*a*Sxy + a**2 * Sxx - 2*theta*(1-a)*(Sy - a*Sx) + n*theta**2 * (1-a)**2) / n
    sigma = np.sqrt(sigmah2*2*kappa / (1-a**2))
    r0 = rates[n-1]
    
    return [kappa, theta, sigma, r0]
In [ ]:
params = VasicekCalibration(yieldsp.loc[:, 'yieldsp'].dropna()/100)

kappa = params[0]
theta = params[1]
sigma = params[2]
r0 = params[3]

years = 1
N = years * 252
t = np.arange(0,N)/252

test_sim = VasicekSim(N, r0, kappa, theta, sigma, 1/252)
plt.plot(t,test_sim)
plt.ylabel("Yield Spread")
plt.xlabel("Years")
plt.show()

Simulating from $r_0$ = last observed value and assuming $\theta = 1.75 \% $, we have generated a sequence of 10 ex-ante trajectories of the yield spread.

In [ ]:
M = 10
rates_arr = VasicekMultiSim(M, N, r0, kappa, theta, sigma)

plt.plot(t,rates_arr)
plt.hlines(y=theta, xmin = -100, xmax=100, zorder=10, linestyles = 'dashed', label='Theta')
plt.annotate('Theta', xy=(1.0, theta + 0.0005))
plt.xlim(-0.05, 1.05)
plt.ylabel("Yield Spread")
plt.xlabel("Years")
plt.title("Long-run (mean reversion level) yield spread, theta = 1.75% ")
plt.show()

Effect of $r_0$

We observe the model's mean reverting nature by specifying $r_0$ further away from $\theta$. Over time this pulls the yield spread towards $\theta$.

In [ ]:
M = 10
rates_arr = VasicekMultiSim(M, N, -0.01, kappa, theta, sigma)
plt.plot(t,rates_arr)
plt.hlines(y=theta, xmin = -100, xmax=100, zorder=10, linestyles = 'dashed', label='Theta')
plt.annotate('Theta', xy=(1.0, theta+0.0005))
plt.xlim(-0.05, 1.05)
plt.ylabel("Yield Spread")
plt.xlabel("Years")
plt.title("Last observed value, r0, is further away from theta to -1% ")
plt.show()

Effect of $\kappa$

The magnitude of $\kappa$ controls the speed of the reversion. As $\kappa$ grows, mean reversion quickens.

In [ ]:
M = 10
rates_arr = VasicekMultiSim(M, N, -0.01, kappa*10, theta, sigma)
plt.plot(t,rates_arr)
plt.hlines(y=theta, xmin = -100, xmax=100, zorder=10, linestyles = 'dashed', label='Theta')
plt.annotate('Theta', xy=(1.0, theta+0.0005))
plt.xlim(-0.05, 1.05)
plt.ylabel("Yield Spread")
plt.xlabel("Years")
plt.title("Kappa (mean reversion speed) scaled by 10 times")
plt.show()

Effect of $\sigma$

Likewise, larger $\sigma$ widens the volality and the potential distribution rate.

In [ ]:
M = 10
rates_arr = VasicekMultiSim(M, N, -0.01, kappa*2, theta, sigma*5)
plt.plot(t,rates_arr)
plt.hlines(y=theta, xmin = -100, xmax=100, zorder=10, linestyles = 'dashed', label='Theta')
plt.annotate('Theta', xy=(1.0, theta+0.0005))
plt.xlim(-0.05, 1.05)
plt.ylabel("Yield Spread")
plt.xlabel("Years")
plt.title("Kappa and sigma (volatility in yield spread) scaled by 2 and 5 times, respectively")
plt.show()

Stationarity

We first check for stationarity of the yield spread using the Augmented Dickey Fuller Test.

In [ ]:
import statsmodels.tsa.api as smt

'''
Test stationarity by:
    - running Augmented Dickey-Fuller test with 95%
    - plotting mean and variance of a sample from data
    - plotting autocorrelation and partial autocorrelation
'''

def test_stationarity_acf_pacf(ts, sample=0.20, maxlag=30, figsize= (18,12)):
    
    # configuring all the plot in one image itself 
    plt.style.context(style='default')
    stationarity_fig = plt.figure(figsize=figsize, dpi = 200)
    layout = (3, 2)
    ts_plot = plt.subplot2grid(layout, loc=(0,0), colspan=2)
    pacf_plot = plt.subplot2grid(layout, loc=(1,0))
    acf_plot = plt.subplot2grid(layout, loc=(1,1))
    qq_plot = plt.subplot2grid(layout, loc=(2, 0))
    pp_plot = plt.subplot2grid(layout, loc=(2, 1))

    ## plot ts with mean/std of a sample from the first x% 
    df_ts = ts.to_frame(name="ts")
    sample_size = int(len(ts)*sample)
    df_ts["mean"] = df_ts["ts"].head(sample_size).mean()
    df_ts["lower_limit"] = df_ts["ts"].head(sample_size).mean() + df_ts["ts"].head(sample_size).std()
    df_ts["upper_limit"] = df_ts["ts"].head(sample_size).mean()  - df_ts["ts"].head(sample_size).std()
    df_ts["ts"].plot(ax=ts_plot, color="black", legend=False)
    df_ts["mean"].plot(ax=ts_plot, legend=False, color="red")
    ts_plot.fill_between(x=df_ts.index, y1=df_ts['lower_limit'], 
            y2=df_ts['upper_limit'], color='paleturquoise', alpha=0.4)
    df_ts["mean"].head(sample_size).plot(ax=ts_plot,
            legend=False, color="red", linewidth=0.9)
    ts_plot.fill_between(x=df_ts.head(sample_size).index, 
                        y1=df_ts['lower_limit'].head(sample_size), 
                        y2=df_ts['upper_limit'].head(sample_size),
                        color='lightskyblue')
    
    ## test stationarity (Augmented Dickey-Fuller)
    adfuller_test = sm.tsa.stattools.adfuller(ts, maxlag=maxlag, autolag="AIC")
    adf, p, critical_value = adfuller_test[0], adfuller_test[1], adfuller_test[4]["5%"]
    p = round(p, 3)
    conclusion = " Stationary" if p < 0.05 else " Non-Stationary"
    ts_plot.set_title('Augmented Dickey-Fuller Test 95% of '+ variable + ':' + conclusion +
                    ' (p value: '+str(p)+')')
    
    ## pacf (for MA) e acf (for AR) 
    smt.graphics.plot_pacf(ts, lags=maxlag, ax = pacf_plot, 
                title="Partial Autocorrelation Function (PACF) of " + variable)
    smt.graphics.plot_acf(ts, lags=maxlag, ax=acf_plot,
                title="Autocorrelation Function (ACF) of "+ variable)
    sm.qqplot(ts, line='s', ax=qq_plot)
    qq_plot.set_title('QQ Plot of ' + variable)        
    scs.probplot(ts, sparams=(ts.mean(), ts.std()), plot=pp_plot)

     
    # plt.suptitle("Stationarity analysis for {}".format(variable))
    plt.tight_layout() 
    plt.show()

Stationarity analysis of yield spread

In [ ]:
for variable in yieldsp.columns:
    test_stationarity_acf_pacf(yieldsp[variable], sample=0.20, maxlag=20) 

A low p-value of 0.007 indicates that yieldsp is stationary; resulting in yieldsp having constant variance and covariance.

The ACF plot shows a strong autocorrelation of lags as spikes very gradually reduce.

The PACF shows a significant lag for perhaps 3 days, with significant lags spotty out to perhaps 20 days. We can try MA(4).

Stationarity analysis of the differenced yield spread

In [ ]:
diff = yieldsp['yieldsp'].diff().to_frame()

yield_diff = diff.dropna(inplace = False).replace([np.inf, -np.inf], 0).dropna(axis=1)

for variable in yield_diff.columns:
    test_stationarity_acf_pacf(yield_diff[variable], sample=0.20, maxlag=20)

From the ACF and PACF plots of the differenced yield curve, we can try AR(3) and MA(3).

ARIMA Models

ARIMA(1,0,3) on yield spread

Fitting an ARMA(1,0,3) model

In [ ]:
X = yieldsp.values
size = int(len(X) * 0.95)
train, test = X[0:size], X[size:len(X)]

# fit model in the whole dataset
model = ARIMA(train, order=(1,0,3))

model_fit_arima = model.fit(disp=0)
print(model_fit_arima.summary())
                              ARMA Model Results                              
==============================================================================
Dep. Variable:                      y   No. Observations:                 9253
Model:                     ARMA(1, 3)   Log Likelihood               10592.283
Method:                       css-mle   S.D. of innovations              0.077
Date:                Sat, 26 Dec 2020   AIC                         -21172.567
Time:                        03:41:19   BIC                         -21129.771
Sample:                             0   HQIC                        -21158.025
                                                                              
==============================================================================
                 coef    std err          z      P>|z|      [0.025      0.975]
------------------------------------------------------------------------------
const          1.7712      0.328      5.401      0.000       1.128       2.414
ar.L1.y        0.9978      0.001   1454.523      0.000       0.996       0.999
ma.L1.y        0.0438      0.010      4.231      0.000       0.024       0.064
ma.L2.y       -0.0211      0.011     -1.956      0.051      -0.042    4.58e-05
ma.L3.y       -0.0696      0.011     -6.319      0.000      -0.091      -0.048
                                    Roots                                    
=============================================================================
                  Real          Imaginary           Modulus         Frequency
-----------------------------------------------------------------------------
AR.1            1.0022           +0.0000j            1.0022            0.0000
MA.1            2.4175           -0.0000j            2.4175           -0.0000
MA.2           -1.3604           -2.0240j            2.4386           -0.3442
MA.3           -1.3604           +2.0240j            2.4386            0.3442
-----------------------------------------------------------------------------

Fitting the ARIMA(1, 0, 3) model yields the following equation with an AIC of − 20, 952.034 :

$$ yieldsp_t = 1.7712 + 0.9978 yieldsp_{t-1} + 0.0438 \epsilon_{t-1} - 0.0211 \epsilon_{t-2} - 0.0696 \epsilon_{t-3} $$

Plotting residuals from ARMA(1,0,3) on yield spread

In [ ]:
# plot residual errors
resid_arima = DataFrame(model_fit_arima.resid, columns = ['resid_arima'])
fig_res = resid_arima.iplot(asFigure=True, kind='scatter', xTitle='Index', yTitle='Residuals')


fig_res.update_layout(
    title={
        'text': "Residuals from ARIMA(1,0,3)",
        'y':0.9,
        'x':0.5,
        'xanchor': 'center',
        'yanchor': 'top'})

fig_res.show()
In [ ]:
hist_data = [resid_arima['resid_arima']]
group_labels = ['distplot'] # name of the dataset

fig_kde = ff.create_distplot(hist_data, group_labels, bin_size=0.01)
fig_kde.update_layout(
    title={
        'text': "Histogram, KDE and a Rug Plot of Residuls from ARIMA(1,0,3)",
        'y':0.9,
        'x':0.5,
        'xanchor': 'center',
        'yanchor': 'top'})

fig_kde.show()
In [ ]:
print(resid_arima.describe())
       resid_arima
count  9253.000000
mean     -0.000066
std       0.077213
min      -1.214803
25%      -0.037550
50%      -0.001489
75%       0.036286
max       0.753097

The plots suggest that the errors are Gaussian with a mean centered at approximately zero (-0.000066), indicating no bias in the predictions. The line plot of residuals suggest that the model captures of the trend information.

Evaluation using walk forward validation

In [ ]:
# we perform a rolling forecast i.e. re-create the ARIMA forecast when a new observation is received. 

X = yieldsp.values
size = int(len(X) * 0.95)
train, test = X[0:size], X[size:len(X)]
history = [x for x in train]
predictions = list()

# walk-forward validation
for t in range(len(test)):
    model = ARIMA(history, order=(1,0,3))
    model_fit = model.fit(disp=0)
    output = model_fit.forecast()
    yhat = output[0]
    predictions.append(yhat)
    obs = test[t]
    history.append(obs)
    print('predicted=%f, expected=%f' % (yhat, obs))
        
predicted=0.154614, expected=0.250000
predicted=0.259821, expected=0.250000
predicted=0.257821, expected=0.270000
predicted=0.267458, expected=0.290000
predicted=0.294739, expected=0.310000
predicted=0.312632, expected=0.280000
predicted=0.280029, expected=0.260000
predicted=0.262132, expected=0.270000
predicted=0.276402, expected=0.300000
predicted=0.305564, expected=0.330000
predicted=0.333266, expected=0.380000
predicted=0.383030, expected=0.310000
predicted=0.307405, expected=0.350000
predicted=0.353337, expected=0.350000
predicted=0.357250, expected=0.370000
predicted=0.370817, expected=0.330000
predicted=0.331424, expected=0.300000
predicted=0.301895, expected=0.280000
predicted=0.285887, expected=0.220000
predicted=0.223226, expected=0.300000
predicted=0.309546, expected=0.310000
predicted=0.316277, expected=0.290000
predicted=0.286803, expected=0.280000
predicted=0.283554, expected=0.230000
predicted=0.233069, expected=0.200000
predicted=0.203641, expected=0.200000
predicted=0.207751, expected=0.250000
predicted=0.257606, expected=0.270000
predicted=0.273248, expected=0.230000
predicted=0.228325, expected=0.230000
predicted=0.233537, expected=0.200000
predicted=0.204996, expected=0.200000
predicted=0.203844, expected=0.240000
predicted=0.247415, expected=0.190000
predicted=0.190577, expected=0.200000
predicted=0.202567, expected=0.190000
predicted=0.196747, expected=0.240000
predicted=0.244877, expected=0.280000
predicted=0.284808, expected=0.320000
predicted=0.321010, expected=0.260000
predicted=0.257501, expected=0.260000
predicted=0.262283, expected=0.220000
predicted=0.225797, expected=0.190000
predicted=0.192637, expected=0.160000
predicted=0.165821, expected=0.180000
predicted=0.187300, expected=0.150000
predicted=0.153911, expected=0.160000
predicted=0.163589, expected=0.180000
predicted=0.186685, expected=0.140000
predicted=0.140766, expected=0.160000
predicted=0.164199, expected=0.150000
predicted=0.155784, expected=0.060000
predicted=0.058506, expected=0.050000
predicted=0.056367, expected=-0.020000
predicted=-0.012606, expected=-0.030000
predicted=-0.024693, expected=-0.050000
predicted=-0.041507, expected=-0.050000
predicted=-0.044724, expected=-0.040000
predicted=-0.033967, expected=0.010000
predicted=0.016202, expected=0.060000
predicted=0.064363, expected=0.060000
predicted=0.059530, expected=0.080000
predicted=0.081594, expected=0.070000
predicted=0.073053, expected=0.060000
predicted=0.061944, expected=0.090000
predicted=0.095945, expected=0.090000
predicted=0.093705, expected=0.050000
predicted=0.049977, expected=0.080000
predicted=0.086292, expected=0.120000
predicted=0.127466, expected=0.120000
predicted=0.120457, expected=0.170000
predicted=0.173453, expected=0.150000
predicted=0.151985, expected=0.150000
predicted=0.150475, expected=0.150000
predicted=0.155174, expected=0.120000
predicted=0.122192, expected=0.090000
predicted=0.092997, expected=0.110000
predicted=0.117454, expected=0.090000
predicted=0.094317, expected=0.100000
predicted=0.103245, expected=0.080000
predicted=0.084424, expected=0.090000
predicted=0.093955, expected=0.080000
predicted=0.084534, expected=0.110000
predicted=0.114594, expected=0.070000
predicted=0.072156, expected=0.020000
predicted=0.020635, expected=0.060000
predicted=0.069580, expected=0.020000
predicted=0.024395, expected=0.040000
predicted=0.042678, expected=-0.010000
predicted=-0.005365, expected=0.010000
predicted=0.014433, expected=-0.050000
predicted=-0.045580, expected=0.000000
predicted=0.006011, expected=0.000000
predicted=0.007058, expected=0.020000
predicted=0.021236, expected=0.040000
predicted=0.044658, expected=0.010000
predicted=0.010954, expected=-0.060000
predicted=-0.059793, expected=-0.030000
predicted=-0.020999, expected=-0.110000
predicted=-0.105562, expected=-0.120000
predicted=-0.116885, expected=-0.160000
predicted=-0.151320, expected=-0.210000
predicted=-0.206521, expected=-0.280000
predicted=-0.274735, expected=-0.230000
predicted=-0.218302, expected=-0.230000
predicted=-0.222159, expected=-0.210000
predicted=-0.208233, expected=-0.190000
predicted=-0.184565, expected=-0.140000
predicted=-0.135297, expected=-0.120000
predicted=-0.117567, expected=-0.110000
predicted=-0.109152, expected=-0.090000
predicted=-0.086487, expected=-0.110000
predicted=-0.108004, expected=-0.140000
predicted=-0.138250, expected=-0.160000
predicted=-0.154626, expected=-0.150000
predicted=-0.143121, expected=-0.130000
predicted=-0.124046, expected=-0.040000
predicted=-0.033137, expected=-0.110000
predicted=-0.112048, expected=-0.120000
predicted=-0.120676, expected=-0.100000
predicted=-0.089679, expected=-0.130000
predicted=-0.127662, expected=-0.120000
predicted=-0.116335, expected=-0.180000
predicted=-0.176062, expected=-0.220000
predicted=-0.217015, expected=-0.250000
predicted=-0.241934, expected=-0.190000
predicted=-0.179958, expected=-0.210000
predicted=-0.205989, expected=-0.190000
predicted=-0.188262, expected=-0.130000
predicted=-0.121773, expected=-0.040000
predicted=-0.034979, expected=-0.020000
predicted=-0.021339, expected=-0.070000
predicted=-0.074284, expected=-0.020000
predicted=-0.013937, expected=-0.080000
predicted=-0.076733, expected=-0.010000
predicted=-0.005821, expected=-0.010000
predicted=-0.003215, expected=-0.040000
predicted=-0.042374, expected=0.020000
predicted=0.027411, expected=-0.050000
predicted=-0.048258, expected=-0.020000
predicted=-0.017803, expected=-0.040000
predicted=-0.032366, expected=-0.060000
predicted=-0.058894, expected=-0.020000
predicted=-0.012482, expected=-0.060000
predicted=-0.057110, expected=-0.170000
predicted=-0.172590, expected=-0.200000
predicted=-0.191512, expected=-0.300000
predicted=-0.292065, expected=-0.320000
predicted=-0.312856, expected=-0.310000
predicted=-0.297556, expected=-0.300000
predicted=-0.294021, expected=-0.260000
predicted=-0.254544, expected=-0.350000
predicted=-0.350355, expected=-0.320000
predicted=-0.314888, expected=-0.370000
predicted=-0.362038, expected=-0.390000
predicted=-0.387857, expected=-0.320000
predicted=-0.308465, expected=-0.340000
predicted=-0.336538, expected=-0.390000
predicted=-0.392027, expected=-0.380000
predicted=-0.371892, expected=-0.380000
predicted=-0.372576, expected=-0.450000
predicted=-0.449573, expected=-0.470000
predicted=-0.464285, expected=-0.490000
predicted=-0.480849, expected=-0.520000
predicted=-0.515224, expected=-0.490000
predicted=-0.481863, expected=-0.490000
predicted=-0.483684, expected=-0.510000
predicted=-0.508212, expected=-0.500000
predicted=-0.494065, expected=-0.400000
predicted=-0.389974, expected=-0.410000
predicted=-0.409015, expected=-0.330000
predicted=-0.328490, expected=-0.230000
predicted=-0.221926, expected=-0.210000
predicted=-0.212916, expected=-0.160000
predicted=-0.160800, expected=-0.060000
predicted=-0.053752, expected=-0.150000
predicted=-0.155917, expected=-0.180000
predicted=-0.182033, expected=-0.150000
predicted=-0.137611, expected=-0.140000
predicted=-0.135212, expected=-0.170000
predicted=-0.169705, expected=-0.220000
predicted=-0.217269, expected=-0.280000
predicted=-0.275171, expected=-0.160000
predicted=-0.146370, expected=-0.130000
predicted=-0.123481, expected=-0.110000
predicted=-0.113835, expected=-0.200000
predicted=-0.201136, expected=-0.170000
predicted=-0.163889, expected=-0.190000
predicted=-0.181860, expected=-0.160000
predicted=-0.156762, expected=-0.190000
predicted=-0.186117, expected=-0.190000
predicted=-0.187039, expected=-0.180000
predicted=-0.173386, expected=-0.100000
predicted=-0.092914, expected=-0.010000
predicted=-0.004756, expected=0.080000
predicted=0.080417, expected=0.100000
predicted=0.096871, expected=0.090000
predicted=0.086931, expected=0.100000
predicted=0.102808, expected=0.100000
predicted=0.103539, expected=0.130000
predicted=0.133699, expected=0.130000
predicted=0.132893, expected=0.120000
predicted=0.121103, expected=0.100000
predicted=0.103048, expected=0.140000
predicted=0.146292, expected=0.200000
predicted=0.206301, expected=0.210000
predicted=0.209769, expected=0.160000
predicted=0.157394, expected=0.150000
predicted=0.153786, expected=0.210000
predicted=0.219265, expected=0.260000
predicted=0.264284, expected=0.300000
predicted=0.299918, expected=0.250000
predicted=0.247457, expected=0.360000
predicted=0.366384, expected=0.390000
predicted=0.395091, expected=0.330000
predicted=0.321954, expected=0.310000
predicted=0.312236, expected=0.250000
predicted=0.255198, expected=0.270000
predicted=0.275857, expected=0.240000
predicted=0.245620, expected=0.220000
predicted=0.221812, expected=0.160000
predicted=0.163638, expected=0.190000
predicted=0.197435, expected=0.190000
predicted=0.196677, expected=0.150000
predicted=0.149648, expected=0.140000
predicted=0.144399, expected=0.150000
predicted=0.156996, expected=0.190000
predicted=0.195246, expected=0.230000
predicted=0.233640, expected=0.150000
predicted=0.146723, expected=0.220000
predicted=0.225675, expected=0.260000
predicted=0.268890, expected=0.310000
predicted=0.309063, expected=0.290000
predicted=0.289044, expected=0.290000
predicted=0.290664, expected=0.220000
predicted=0.221441, expected=0.340000
predicted=0.349460, expected=0.250000
predicted=0.251374, expected=0.320000
predicted=0.319723, expected=0.330000
predicted=0.338947, expected=0.360000
predicted=0.358879, expected=0.350000
predicted=0.351442, expected=0.340000
predicted=0.341201, expected=0.340000
predicted=0.343765, expected=0.320000
predicted=0.322808, expected=0.320000
predicted=0.323446, expected=0.310000
predicted=0.314154, expected=0.330000
predicted=0.334129, expected=0.370000
predicted=0.375064, expected=0.340000
predicted=0.339650, expected=0.280000
predicted=0.278738, expected=0.250000
predicted=0.255528, expected=0.290000
predicted=0.299250, expected=0.330000
predicted=0.335604, expected=0.310000
predicted=0.308911, expected=0.290000
predicted=0.290624, expected=0.280000
predicted=0.284775, expected=0.250000
predicted=0.253163, expected=0.220000
predicted=0.223195, expected=0.260000
predicted=0.267765, expected=0.280000
predicted=0.285156, expected=0.220000
predicted=0.217586, expected=0.220000
predicted=0.223746, expected=0.190000
predicted=0.196258, expected=0.160000
predicted=0.162243, expected=0.060000
predicted=0.062144, expected=0.080000
predicted=0.088762, expected=0.040000
predicted=0.048141, expected=0.000000
predicted=0.001239, expected=-0.040000
predicted=-0.033796, expected=-0.030000
predicted=-0.022062, expected=0.040000
predicted=0.048941, expected=0.090000
predicted=0.093662, expected=0.080000
predicted=0.077703, expected=0.030000
predicted=0.028878, expected=-0.020000
predicted=-0.016606, expected=0.020000
predicted=0.029389, expected=0.040000
predicted=0.046601, expected=0.020000
predicted=0.019617, expected=0.010000
predicted=0.012918, expected=-0.030000
predicted=-0.026202, expected=-0.020000
predicted=-0.014620, expected=-0.060000
predicted=-0.055442, expected=-0.100000
predicted=-0.097719, expected=-0.150000
predicted=-0.144404, expected=-0.200000
predicted=-0.194373, expected=-0.200000
predicted=-0.191619, expected=-0.150000
predicted=-0.140435, expected=-0.140000
predicted=-0.136646, expected=-0.030000
predicted=-0.024711, expected=0.070000
predicted=0.075366, expected=0.300000
predicted=0.303599, expected=0.300000
predicted=0.291927, expected=0.290000
predicted=0.277430, expected=0.210000
predicted=0.210517, expected=0.320000
predicted=0.329184, expected=0.400000
predicted=0.408489, expected=0.550000
predicted=0.549757, expected=0.660000
predicted=0.659600, expected=0.490000
predicted=0.473207, expected=0.830000
predicted=0.842865, expected=1.160000
predicted=1.180067, expected=1.080000
predicted=1.046485, expected=0.870000
predicted=0.844447, expected=0.740000
predicted=0.748126, expected=0.830000
predicted=0.850094, expected=0.880000
predicted=0.888750, expected=0.830000
predicted=0.823027, expected=0.690000
predicted=0.685466, expected=0.580000
predicted=0.584662, expected=0.590000
predicted=0.604247, expected=0.530000
predicted=0.536466, expected=0.540000
predicted=0.543926, expected=0.520000
predicted=0.526622, expected=0.520000
predicted=0.522565, expected=0.610000
predicted=0.618188, expected=0.550000
predicted=0.548081, expected=0.480000
predicted=0.474973, expected=0.500000
predicted=0.509967, expected=0.560000
predicted=0.568987, expected=0.490000
predicted=0.486285, expected=0.470000
predicted=0.470179, expected=0.530000
predicted=0.541117, expected=0.510000
predicted=0.511056, expected=0.470000
predicted=0.467364, expected=0.510000
predicted=0.517586, expected=0.500000
predicted=0.503768, expected=0.480000
predicted=0.479029, expected=0.550000
predicted=0.557469, expected=0.510000
predicted=0.510586, expected=0.530000
predicted=0.529562, expected=0.550000
predicted=0.556321, expected=0.520000
predicted=0.519175, expected=0.510000
predicted=0.511576, expected=0.530000
predicted=0.536114, expected=0.600000
predicted=0.605546, expected=0.520000
predicted=0.516087, expected=0.570000
predicted=0.572346, expected=0.610000
predicted=0.618846, expected=0.560000
predicted=0.555333, expected=0.510000
predicted=0.509265, expected=0.510000
predicted=0.517700, expected=0.520000
predicted=0.525808, expected=0.600000
predicted=0.605660, expected=0.570000
predicted=0.569112, expected=0.560000
predicted=0.557752, expected=0.560000
predicted=0.565276, expected=0.540000
predicted=0.541996, expected=0.550000
predicted=0.553285, expected=0.530000
predicted=0.533096, expected=0.550000
predicted=0.553240, expected=0.510000
predicted=0.511908, expected=0.520000
predicted=0.522722, expected=0.530000
predicted=0.535703, expected=0.610000
predicted=0.615016, expected=0.670000
predicted=0.672633, expected=0.760000
predicted=0.759709, expected=0.710000
predicted=0.704275, expected=0.650000
predicted=0.644922, expected=0.580000
predicted=0.584180, expected=0.490000
predicted=0.493572, expected=0.550000
predicted=0.561613, expected=0.530000
predicted=0.536453, expected=0.580000
predicted=0.581183, expected=0.570000
predicted=0.573235, expected=0.550000
predicted=0.548696, expected=0.550000
predicted=0.553859, expected=0.550000
predicted=0.553933, expected=0.560000
predicted=0.562765, expected=0.540000
predicted=0.541653, expected=0.520000
predicted=0.521676, expected=0.500000
predicted=0.503692, expected=0.500000
predicted=0.504421, expected=0.500000
predicted=0.504000, expected=0.550000
predicted=0.554933, expected=0.540000
predicted=0.541176, expected=0.540000
predicted=0.539608, expected=0.500000
predicted=0.501897, expected=0.520000
predicted=0.524322, expected=0.490000
predicted=0.493452, expected=0.520000
predicted=0.523242, expected=0.500000
predicted=0.503381, expected=0.480000
predicted=0.480253, expected=0.480000
predicted=0.484755, expected=0.510000
predicted=0.515351, expected=0.530000
predicted=0.532677, expected=0.490000
predicted=0.488638, expected=0.480000
predicted=0.482159, expected=0.470000
predicted=0.475272, expected=0.470000
predicted=0.473288, expected=0.480000
predicted=0.483902, expected=0.510000
predicted=0.513975, expected=0.480000
predicted=0.480085, expected=0.470000
predicted=0.471125, expected=0.460000
predicted=0.464760, expected=0.460000
predicted=0.463405, expected=0.460000
predicted=0.463399, expected=0.430000
predicted=0.431639, expected=0.450000
predicted=0.454475, expected=0.450000
predicted=0.454419, expected=0.470000
predicted=0.472170, expected=0.480000
predicted=0.482959, expected=0.530000
predicted=0.533393, expected=0.580000
predicted=0.582973, expected=0.610000
predicted=0.609320, expected=0.610000
predicted=0.608589, expected=0.590000
predicted=0.589692, expected=0.580000
predicted=0.582359, expected=0.570000
predicted=0.573407, expected=0.540000
predicted=0.541958, expected=0.540000
predicted=0.544009, expected=0.530000
predicted=0.534273, expected=0.580000
predicted=0.584929, expected=0.580000
predicted=0.582206, expected=0.630000
predicted=0.631408, expected=0.640000
predicted=0.642022, expected=0.610000
predicted=0.607429, expected=0.560000
predicted=0.560436, expected=0.540000
predicted=0.544847, expected=0.520000
predicted=0.525178, expected=0.610000
predicted=0.618143, expected=0.560000
predicted=0.559775, expected=0.590000
predicted=0.589100, expected=0.560000
predicted=0.564579, expected=0.560000
predicted=0.560781, expected=0.570000
predicted=0.574991, expected=0.570000
predicted=0.572349, expected=0.570000
predicted=0.571808, expected=0.600000
predicted=0.604052, expected=0.600000
predicted=0.601769, expected=0.580000
predicted=0.579575, expected=0.580000
predicted=0.583197, expected=0.570000
predicted=0.573366, expected=0.570000
predicted=0.572549, expected=0.560000
predicted=0.562897, expected=0.560000
predicted=0.562839, expected=0.570000
predicted=0.573703, expected=0.590000
predicted=0.593184, expected=0.590000
predicted=0.591414, expected=0.610000
predicted=0.612130, expected=0.680000
predicted=0.685094, expected=0.660000
predicted=0.658394, expected=0.710000
predicted=0.710285, expected=0.690000
predicted=0.691951, expected=0.690000
predicted=0.688957, expected=0.630000
predicted=0.631148, expected=0.610000
predicted=0.612814, expected=0.630000
predicted=0.637662, expected=0.650000
predicted=0.653947, expected=0.670000
predicted=0.671501, expected=0.710000
predicted=0.712687, expected=0.730000
predicted=0.730961, expected=0.780000
predicted=0.781171, expected=0.750000
predicted=0.748440, expected=0.700000
predicted=0.697260, expected=0.690000
predicted=0.695093, expected=0.690000
predicted=0.695516, expected=0.760000
predicted=0.765570, expected=0.790000
predicted=0.792082, expected=0.780000
predicted=0.776458, expected=0.800000
predicted=0.801594, expected=0.680000
predicted=0.677130, expected=0.690000
predicted=0.693726, expected=0.730000
predicted=0.741969, expected=0.850000
predicted=0.855045, expected=0.880000
predicted=0.878099, expected=0.780000
predicted=0.769518, expected=0.800000
predicted=0.803706, expected=0.820000
predicted=0.828896, expected=0.780000
predicted=0.777354, expected=0.790000
predicted=0.792484, expected=0.790000
predicted=0.795044, expected=0.760000
predicted=0.759666, expected=0.780000
predicted=0.783863, expected=0.790000
predicted=0.794290, expected=0.790000
predicted=0.790256, expected=0.750000
predicted=0.749946, expected=0.760000
predicted=0.763681, expected=0.830000
predicted=0.837487, expected=0.860000
predicted=0.860731, expected=0.840000
predicted=0.835845, expected=0.880000
predicted=0.882671, expected=0.860000
predicted=0.861344, expected=0.830000
predicted=0.827924, expected=0.870000
predicted=0.875987, expected=0.840000
predicted=0.841573, expected=0.820000
predicted=0.818819, expected=0.810000
predicted=0.814550, expected=0.840000
In [ ]:
pred = pd.DataFrame(predictions, columns = ['pred'])
print("Shape of predictions dataframe is {}".format(pred.shape))

#yieldsp.iloc[-len(test):,] 
Shape of predictions dataframe is (488, 1)
In [ ]:
pred.index = pd.date_range(start="2019-08-16",end="2020-12-15" )
test = pd.DataFrame(test, columns = ['test_yieldsp'])
# print(pred.shape)
test.index = pd.date_range(start="2019-08-16",end="2020-12-15")
# print(test.shape)
# combine the predictions from leveled ARIMA with the test values
merge = pd.merge(pred, test, how='inner', left_index=True, right_index=True)
merge.tail()
# print(merge.shape)
Out[ ]:
pred test_yieldsp
2020-12-11 0.827924 0.87
2020-12-12 0.875987 0.84
2020-12-13 0.841573 0.82
2020-12-14 0.818819 0.81
2020-12-15 0.814550 0.84
In [ ]:
error = sqrt(mean_squared_error(merge['test_yieldsp'], merge['pred']))
print('Test RMSE from the leveled model - ARIMA(1,0,3): %.5f' % error)
Test RMSE from the leveled model - ARIMA(1,0,3): 0.05110

Thus, the final RMSE for the ARMA(1,0,3) model is 0.05110.

ARIMA(3,1,3) on the differenced yield spread

Fitting an ARIMA(3,1,3) model on yield_diff

In [ ]:
X_diff = yield_diff.values
size = int(len(X_diff) * 0.95)
train_diff, test_diff = X_diff[0:size], X_diff[size:len(X_diff)]
history = [x for x in train_diff]
predictions = list()

# fit model
model_diff = ARIMA(train_diff, order=(3,1,3))
mod_fit_diff = model_diff.fit(disp=0)
print(mod_fit_diff.summary())
                             ARIMA Model Results                              
==============================================================================
Dep. Variable:                    D.y   No. Observations:                 9252
Model:                 ARIMA(3, 1, 3)   Log Likelihood               10578.756
Method:                       css-mle   S.D. of innovations              0.077
Date:                Sat, 26 Dec 2020   AIC                         -21141.512
Time:                        04:01:56   BIC                         -21084.451
Sample:                             1   HQIC                        -21122.122
                                                                              
==============================================================================
                 coef    std err          z      P>|z|      [0.025      0.975]
------------------------------------------------------------------------------
const      -7.392e-08    2.8e-07     -0.264      0.791   -6.22e-07    4.74e-07
ar.L1.D.y      0.1647      0.009     18.811      0.000       0.148       0.182
ar.L2.D.y      0.1576      0.039      4.045      0.000       0.081       0.234
ar.L3.D.y     -0.0594      0.010     -5.657      0.000      -0.080      -0.039
ma.L1.D.y     -1.1186        nan        nan        nan         nan         nan
ma.L2.D.y     -0.0759      0.038     -2.014      0.044      -0.150      -0.002
ma.L3.D.y      0.1945      0.038      5.103      0.000       0.120       0.269
                                    Roots                                    
=============================================================================
                  Real          Imaginary           Modulus         Frequency
-----------------------------------------------------------------------------
AR.1           -2.1763           -0.0000j            2.1763           -0.5000
AR.2            2.4153           -1.3803j            2.7819           -0.0826
AR.3            2.4153           +1.3803j            2.7819            0.0826
MA.1            1.0000           +0.0000j            1.0000            0.0000
MA.2           -2.5927           +0.0000j            2.5927            0.5000
MA.3            1.9827           +0.0000j            1.9827            0.0000
-----------------------------------------------------------------------------

Fitting the ARIMA(3,1, 3) model on the differenced yield spread yields the following equation with an AIC of − 21141.512 :

$$ \Delta yieldsp_t = -7.392*10^{-8} + 0.1647 \Delta yieldsp_{t-1} + 0.1576 \Delta yieldsp_{t-2} -0.0594 \Delta yieldsp_{t-3} - 1.1186 \epsilon_{t-1} - 0.0759 \epsilon_{t-2} + 0.1945 \epsilon_{t-3} $$

Plotting residuals from ARMA(3,1,3) on yield spread

In [ ]:
# plot residual errors
resid_diff = DataFrame(mod_fit_diff.resid, columns = ['resid_diff'])
fig_res = resid_diff.iplot(asFigure=True, kind='scatter', xTitle='Index', yTitle='Residuals')
fig_res.update_layout(
    title={
        'text': "Residuals from Differenced ARIMA(3,1,3)",
        'y':0.9,
        'x':0.5,
        'xanchor': 'center',
        'yanchor': 'top'})

fig_res.show()
In [ ]:
hist_data = [resid_diff['resid_diff']]
group_labels = ['distplot'] # name of the dataset

fig_kde = ff.create_distplot(hist_data, group_labels, bin_size=0.01)
fig_kde.update_layout(
    title={
        'text': "Histogram, KDE and a Rug Plot of Residuls from Differenced ARIMA(3,1,3)",
        'y':0.9,
        'x':0.5,
        'xanchor': 'center',
        'yanchor': 'top'})

fig_kde.show()
In [ ]:
print(resid_diff.describe())
        resid_diff
count  9252.000000
mean      0.000555
std       0.077457
min      -1.136089
25%      -0.037421
50%      -0.000781
75%       0.037110
max       0.750472

Evaluation using walk forward validation

In [ ]:
# we perform a rolling forecast i.e. re-create the ARIMA forecast when a new observation is received. 

X = yieldsp['yieldsp'].values
size = int(len(X) * 0.95)
train_diff, test_diff = X[0:size], X[size:len(X)]
history = [x for x in train]
predictions_diff = list()
gap = 1

def difference(dataset, interval=1):
	diff = list()
	for i in range(interval, len(dataset)):
		value = dataset[i] - dataset[i - interval]
		diff.append(value)
	return np.array(diff)

diff = difference(X, 1)

def invert_diff(history, yhat, interval=1):
    return yhat + history[-interval]

# walk-forward validation
for t in range(len(test)):
    model_diff = ARIMA(diff, order=(3,1,3))
    model_diff_fit = model_diff.fit(disp=0)
    output = model_diff_fit.forecast()
    yhat = output[0]
    inverted = invert_diff(history, yhat, gap)
    predictions_diff.append(inverted)
    obs = test_diff[t]
    history.append(obs)
    print('predicted=%f, expected=%f' % (inverted, obs))
predicted=0.152550, expected=0.250000
predicted=0.252550, expected=0.250000
predicted=0.252550, expected=0.270000
predicted=0.272550, expected=0.290000
predicted=0.292550, expected=0.310000
predicted=0.312550, expected=0.280000
predicted=0.282550, expected=0.260000
predicted=0.262550, expected=0.270000
predicted=0.272550, expected=0.300000
predicted=0.302550, expected=0.330000
predicted=0.332550, expected=0.380000
predicted=0.382550, expected=0.310000
predicted=0.312550, expected=0.350000
predicted=0.352550, expected=0.350000
predicted=0.352550, expected=0.370000
predicted=0.372550, expected=0.330000
predicted=0.332550, expected=0.300000
predicted=0.302550, expected=0.280000
predicted=0.282550, expected=0.220000
predicted=0.222550, expected=0.300000
predicted=0.302550, expected=0.310000
predicted=0.312550, expected=0.290000
predicted=0.292550, expected=0.280000
predicted=0.282550, expected=0.230000
predicted=0.232550, expected=0.200000
predicted=0.202550, expected=0.200000
predicted=0.202550, expected=0.250000
predicted=0.252550, expected=0.270000
predicted=0.272550, expected=0.230000
predicted=0.232550, expected=0.230000
predicted=0.232550, expected=0.200000
predicted=0.202550, expected=0.200000
predicted=0.202550, expected=0.240000
predicted=0.242550, expected=0.190000
predicted=0.192550, expected=0.200000
predicted=0.202550, expected=0.190000
predicted=0.192550, expected=0.240000
predicted=0.242550, expected=0.280000
predicted=0.282550, expected=0.320000
predicted=0.322550, expected=0.260000
predicted=0.262550, expected=0.260000
predicted=0.262550, expected=0.220000
predicted=0.222550, expected=0.190000
predicted=0.192550, expected=0.160000
predicted=0.162550, expected=0.180000
predicted=0.182550, expected=0.150000
predicted=0.152550, expected=0.160000
predicted=0.162550, expected=0.180000
predicted=0.182550, expected=0.140000
predicted=0.142550, expected=0.160000
predicted=0.162550, expected=0.150000
predicted=0.152550, expected=0.060000
predicted=0.062550, expected=0.050000
predicted=0.052550, expected=-0.020000
predicted=-0.017450, expected=-0.030000
predicted=-0.027450, expected=-0.050000
predicted=-0.047450, expected=-0.050000
predicted=-0.047450, expected=-0.040000
predicted=-0.037450, expected=0.010000
predicted=0.012550, expected=0.060000
predicted=0.062550, expected=0.060000
predicted=0.062550, expected=0.080000
predicted=0.082550, expected=0.070000
predicted=0.072550, expected=0.060000
predicted=0.062550, expected=0.090000
predicted=0.092550, expected=0.090000
predicted=0.092550, expected=0.050000
predicted=0.052550, expected=0.080000
predicted=0.082550, expected=0.120000
predicted=0.122550, expected=0.120000
predicted=0.122550, expected=0.170000
predicted=0.172550, expected=0.150000
predicted=0.152550, expected=0.150000
predicted=0.152550, expected=0.150000
predicted=0.152550, expected=0.120000
predicted=0.122550, expected=0.090000
predicted=0.092550, expected=0.110000
predicted=0.112550, expected=0.090000
predicted=0.092550, expected=0.100000
predicted=0.102550, expected=0.080000
predicted=0.082550, expected=0.090000
predicted=0.092550, expected=0.080000
predicted=0.082550, expected=0.110000
predicted=0.112550, expected=0.070000
predicted=0.072550, expected=0.020000
predicted=0.022550, expected=0.060000
predicted=0.062550, expected=0.020000
predicted=0.022550, expected=0.040000
predicted=0.042550, expected=-0.010000
predicted=-0.007450, expected=0.010000
predicted=0.012550, expected=-0.050000
predicted=-0.047450, expected=0.000000
predicted=0.002550, expected=0.000000
predicted=0.002550, expected=0.020000
predicted=0.022550, expected=0.040000
predicted=0.042550, expected=0.010000
predicted=0.012550, expected=-0.060000
predicted=-0.057450, expected=-0.030000
predicted=-0.027450, expected=-0.110000
predicted=-0.107450, expected=-0.120000
predicted=-0.117450, expected=-0.160000
predicted=-0.157450, expected=-0.210000
predicted=-0.207450, expected=-0.280000
predicted=-0.277450, expected=-0.230000
predicted=-0.227450, expected=-0.230000
predicted=-0.227450, expected=-0.210000
predicted=-0.207450, expected=-0.190000
predicted=-0.187450, expected=-0.140000
predicted=-0.137450, expected=-0.120000
predicted=-0.117450, expected=-0.110000
predicted=-0.107450, expected=-0.090000
predicted=-0.087450, expected=-0.110000
predicted=-0.107450, expected=-0.140000
predicted=-0.137450, expected=-0.160000
predicted=-0.157450, expected=-0.150000
predicted=-0.147450, expected=-0.130000
predicted=-0.127450, expected=-0.040000
predicted=-0.037450, expected=-0.110000
predicted=-0.107450, expected=-0.120000
predicted=-0.117450, expected=-0.100000
predicted=-0.097450, expected=-0.130000
predicted=-0.127450, expected=-0.120000
predicted=-0.117450, expected=-0.180000
predicted=-0.177450, expected=-0.220000
predicted=-0.217450, expected=-0.250000
predicted=-0.247450, expected=-0.190000
predicted=-0.187450, expected=-0.210000
predicted=-0.207450, expected=-0.190000
predicted=-0.187450, expected=-0.130000
predicted=-0.127450, expected=-0.040000
predicted=-0.037450, expected=-0.020000
predicted=-0.017450, expected=-0.070000
predicted=-0.067450, expected=-0.020000
predicted=-0.017450, expected=-0.080000
predicted=-0.077450, expected=-0.010000
predicted=-0.007450, expected=-0.010000
predicted=-0.007450, expected=-0.040000
predicted=-0.037450, expected=0.020000
predicted=0.022550, expected=-0.050000
predicted=-0.047450, expected=-0.020000
predicted=-0.017450, expected=-0.040000
predicted=-0.037450, expected=-0.060000
predicted=-0.057450, expected=-0.020000
predicted=-0.017450, expected=-0.060000
predicted=-0.057450, expected=-0.170000
predicted=-0.167450, expected=-0.200000
predicted=-0.197450, expected=-0.300000
predicted=-0.297450, expected=-0.320000
predicted=-0.317450, expected=-0.310000
predicted=-0.307450, expected=-0.300000
predicted=-0.297450, expected=-0.260000
predicted=-0.257450, expected=-0.350000
predicted=-0.347450, expected=-0.320000
predicted=-0.317450, expected=-0.370000
predicted=-0.367450, expected=-0.390000
predicted=-0.387450, expected=-0.320000
predicted=-0.317450, expected=-0.340000
predicted=-0.337450, expected=-0.390000
predicted=-0.387450, expected=-0.380000
predicted=-0.377450, expected=-0.380000
predicted=-0.377450, expected=-0.450000
predicted=-0.447450, expected=-0.470000
predicted=-0.467450, expected=-0.490000
predicted=-0.487450, expected=-0.520000
predicted=-0.517450, expected=-0.490000
predicted=-0.487450, expected=-0.490000
predicted=-0.487450, expected=-0.510000
predicted=-0.507450, expected=-0.500000
predicted=-0.497450, expected=-0.400000
predicted=-0.397450, expected=-0.410000
predicted=-0.407450, expected=-0.330000
predicted=-0.327450, expected=-0.230000
predicted=-0.227450, expected=-0.210000
predicted=-0.207450, expected=-0.160000
predicted=-0.157450, expected=-0.060000
predicted=-0.057450, expected=-0.150000
predicted=-0.147450, expected=-0.180000
predicted=-0.177450, expected=-0.150000
predicted=-0.147450, expected=-0.140000
predicted=-0.137450, expected=-0.170000
predicted=-0.167450, expected=-0.220000
predicted=-0.217450, expected=-0.280000
predicted=-0.277450, expected=-0.160000
predicted=-0.157450, expected=-0.130000
predicted=-0.127450, expected=-0.110000
predicted=-0.107450, expected=-0.200000
predicted=-0.197450, expected=-0.170000
predicted=-0.167450, expected=-0.190000
predicted=-0.187450, expected=-0.160000
predicted=-0.157450, expected=-0.190000
predicted=-0.187450, expected=-0.190000
predicted=-0.187450, expected=-0.180000
predicted=-0.177450, expected=-0.100000
predicted=-0.097450, expected=-0.010000
predicted=-0.007450, expected=0.080000
predicted=0.082550, expected=0.100000
predicted=0.102550, expected=0.090000
predicted=0.092550, expected=0.100000
predicted=0.102550, expected=0.100000
predicted=0.102550, expected=0.130000
predicted=0.132550, expected=0.130000
predicted=0.132550, expected=0.120000
predicted=0.122550, expected=0.100000
predicted=0.102550, expected=0.140000
predicted=0.142550, expected=0.200000
predicted=0.202550, expected=0.210000
predicted=0.212550, expected=0.160000
predicted=0.162550, expected=0.150000
predicted=0.152550, expected=0.210000
predicted=0.212550, expected=0.260000
predicted=0.262550, expected=0.300000
predicted=0.302550, expected=0.250000
predicted=0.252550, expected=0.360000
predicted=0.362550, expected=0.390000
predicted=0.392550, expected=0.330000
predicted=0.332550, expected=0.310000
predicted=0.312550, expected=0.250000
predicted=0.252550, expected=0.270000
predicted=0.272550, expected=0.240000
predicted=0.242550, expected=0.220000
predicted=0.222550, expected=0.160000
predicted=0.162550, expected=0.190000
predicted=0.192550, expected=0.190000
predicted=0.192550, expected=0.150000
predicted=0.152550, expected=0.140000
predicted=0.142550, expected=0.150000
predicted=0.152550, expected=0.190000
predicted=0.192550, expected=0.230000
predicted=0.232550, expected=0.150000
predicted=0.152550, expected=0.220000
predicted=0.222550, expected=0.260000
predicted=0.262550, expected=0.310000
predicted=0.312550, expected=0.290000
predicted=0.292550, expected=0.290000
predicted=0.292550, expected=0.220000
predicted=0.222550, expected=0.340000
predicted=0.342550, expected=0.250000
predicted=0.252550, expected=0.320000
predicted=0.322550, expected=0.330000
predicted=0.332550, expected=0.360000
predicted=0.362550, expected=0.350000
predicted=0.352550, expected=0.340000
predicted=0.342550, expected=0.340000
predicted=0.342550, expected=0.320000
predicted=0.322550, expected=0.320000
predicted=0.322550, expected=0.310000
predicted=0.312550, expected=0.330000
predicted=0.332550, expected=0.370000
predicted=0.372550, expected=0.340000
predicted=0.342550, expected=0.280000
predicted=0.282550, expected=0.250000
predicted=0.252550, expected=0.290000
predicted=0.292550, expected=0.330000
predicted=0.332550, expected=0.310000
predicted=0.312550, expected=0.290000
predicted=0.292550, expected=0.280000
predicted=0.282550, expected=0.250000
predicted=0.252550, expected=0.220000
predicted=0.222550, expected=0.260000
predicted=0.262550, expected=0.280000
predicted=0.282550, expected=0.220000
predicted=0.222550, expected=0.220000
predicted=0.222550, expected=0.190000
predicted=0.192550, expected=0.160000
predicted=0.162550, expected=0.060000
predicted=0.062550, expected=0.080000
predicted=0.082550, expected=0.040000
predicted=0.042550, expected=0.000000
predicted=0.002550, expected=-0.040000
predicted=-0.037450, expected=-0.030000
predicted=-0.027450, expected=0.040000
predicted=0.042550, expected=0.090000
predicted=0.092550, expected=0.080000
predicted=0.082550, expected=0.030000
predicted=0.032550, expected=-0.020000
predicted=-0.017450, expected=0.020000
predicted=0.022550, expected=0.040000
predicted=0.042550, expected=0.020000
predicted=0.022550, expected=0.010000
predicted=0.012550, expected=-0.030000
predicted=-0.027450, expected=-0.020000
predicted=-0.017450, expected=-0.060000
predicted=-0.057450, expected=-0.100000
predicted=-0.097450, expected=-0.150000
predicted=-0.147450, expected=-0.200000
predicted=-0.197450, expected=-0.200000
predicted=-0.197450, expected=-0.150000
predicted=-0.147450, expected=-0.140000
predicted=-0.137450, expected=-0.030000
predicted=-0.027450, expected=0.070000
predicted=0.072550, expected=0.300000
predicted=0.302550, expected=0.300000
predicted=0.302550, expected=0.290000
predicted=0.292550, expected=0.210000
predicted=0.212550, expected=0.320000
predicted=0.322550, expected=0.400000
predicted=0.402550, expected=0.550000
predicted=0.552550, expected=0.660000
predicted=0.662550, expected=0.490000
predicted=0.492550, expected=0.830000
predicted=0.832550, expected=1.160000
predicted=1.162550, expected=1.080000
predicted=1.082550, expected=0.870000
predicted=0.872550, expected=0.740000
predicted=0.742550, expected=0.830000
predicted=0.832550, expected=0.880000
predicted=0.882550, expected=0.830000
predicted=0.832550, expected=0.690000
predicted=0.692550, expected=0.580000
predicted=0.582550, expected=0.590000
predicted=0.592550, expected=0.530000
predicted=0.532550, expected=0.540000
predicted=0.542550, expected=0.520000
predicted=0.522550, expected=0.520000
predicted=0.522550, expected=0.610000
predicted=0.612550, expected=0.550000
predicted=0.552550, expected=0.480000
predicted=0.482550, expected=0.500000
predicted=0.502550, expected=0.560000
predicted=0.562550, expected=0.490000
predicted=0.492550, expected=0.470000
predicted=0.472550, expected=0.530000
predicted=0.532550, expected=0.510000
predicted=0.512550, expected=0.470000
predicted=0.472550, expected=0.510000
predicted=0.512550, expected=0.500000
predicted=0.502550, expected=0.480000
predicted=0.482550, expected=0.550000
predicted=0.552550, expected=0.510000
predicted=0.512550, expected=0.530000
predicted=0.532550, expected=0.550000
predicted=0.552550, expected=0.520000
predicted=0.522550, expected=0.510000
predicted=0.512550, expected=0.530000
predicted=0.532550, expected=0.600000
predicted=0.602550, expected=0.520000
predicted=0.522550, expected=0.570000
predicted=0.572550, expected=0.610000
predicted=0.612550, expected=0.560000
predicted=0.562550, expected=0.510000
predicted=0.512550, expected=0.510000
predicted=0.512550, expected=0.520000
predicted=0.522550, expected=0.600000
predicted=0.602550, expected=0.570000
predicted=0.572550, expected=0.560000
predicted=0.562550, expected=0.560000
predicted=0.562550, expected=0.540000
predicted=0.542550, expected=0.550000
predicted=0.552550, expected=0.530000
predicted=0.532550, expected=0.550000
predicted=0.552550, expected=0.510000
predicted=0.512550, expected=0.520000
predicted=0.522550, expected=0.530000
predicted=0.532550, expected=0.610000
predicted=0.612550, expected=0.670000
predicted=0.672550, expected=0.760000
predicted=0.762550, expected=0.710000
predicted=0.712550, expected=0.650000
predicted=0.652550, expected=0.580000
predicted=0.582550, expected=0.490000
predicted=0.492550, expected=0.550000
predicted=0.552550, expected=0.530000
predicted=0.532550, expected=0.580000
predicted=0.582550, expected=0.570000
predicted=0.572550, expected=0.550000
predicted=0.552550, expected=0.550000
predicted=0.552550, expected=0.550000
predicted=0.552550, expected=0.560000
predicted=0.562550, expected=0.540000
predicted=0.542550, expected=0.520000
predicted=0.522550, expected=0.500000
predicted=0.502550, expected=0.500000
predicted=0.502550, expected=0.500000
predicted=0.502550, expected=0.550000
predicted=0.552550, expected=0.540000
predicted=0.542550, expected=0.540000
predicted=0.542550, expected=0.500000
predicted=0.502550, expected=0.520000
predicted=0.522550, expected=0.490000
predicted=0.492550, expected=0.520000
predicted=0.522550, expected=0.500000
predicted=0.502550, expected=0.480000
predicted=0.482550, expected=0.480000
predicted=0.482550, expected=0.510000
predicted=0.512550, expected=0.530000
predicted=0.532550, expected=0.490000
predicted=0.492550, expected=0.480000
predicted=0.482550, expected=0.470000
predicted=0.472550, expected=0.470000
predicted=0.472550, expected=0.480000
predicted=0.482550, expected=0.510000
predicted=0.512550, expected=0.480000
predicted=0.482550, expected=0.470000
predicted=0.472550, expected=0.460000
predicted=0.462550, expected=0.460000
predicted=0.462550, expected=0.460000
predicted=0.462550, expected=0.430000
predicted=0.432550, expected=0.450000
predicted=0.452550, expected=0.450000
predicted=0.452550, expected=0.470000
predicted=0.472550, expected=0.480000
predicted=0.482550, expected=0.530000
predicted=0.532550, expected=0.580000
predicted=0.582550, expected=0.610000
predicted=0.612550, expected=0.610000
predicted=0.612550, expected=0.590000
predicted=0.592550, expected=0.580000
predicted=0.582550, expected=0.570000
predicted=0.572550, expected=0.540000
predicted=0.542550, expected=0.540000
predicted=0.542550, expected=0.530000
predicted=0.532550, expected=0.580000
predicted=0.582550, expected=0.580000
predicted=0.582550, expected=0.630000
predicted=0.632550, expected=0.640000
predicted=0.642550, expected=0.610000
predicted=0.612550, expected=0.560000
predicted=0.562550, expected=0.540000
predicted=0.542550, expected=0.520000
predicted=0.522550, expected=0.610000
predicted=0.612550, expected=0.560000
predicted=0.562550, expected=0.590000
predicted=0.592550, expected=0.560000
predicted=0.562550, expected=0.560000
predicted=0.562550, expected=0.570000
predicted=0.572550, expected=0.570000
predicted=0.572550, expected=0.570000
predicted=0.572550, expected=0.600000
predicted=0.602550, expected=0.600000
predicted=0.602550, expected=0.580000
predicted=0.582550, expected=0.580000
predicted=0.582550, expected=0.570000
predicted=0.572550, expected=0.570000
predicted=0.572550, expected=0.560000
predicted=0.562550, expected=0.560000
predicted=0.562550, expected=0.570000
predicted=0.572550, expected=0.590000
predicted=0.592550, expected=0.590000
predicted=0.592550, expected=0.610000
predicted=0.612550, expected=0.680000
predicted=0.682550, expected=0.660000
predicted=0.662550, expected=0.710000
predicted=0.712550, expected=0.690000
predicted=0.692550, expected=0.690000
predicted=0.692550, expected=0.630000
predicted=0.632550, expected=0.610000
predicted=0.612550, expected=0.630000
predicted=0.632550, expected=0.650000
predicted=0.652550, expected=0.670000
predicted=0.672550, expected=0.710000
predicted=0.712550, expected=0.730000
predicted=0.732550, expected=0.780000
predicted=0.782550, expected=0.750000
predicted=0.752550, expected=0.700000
predicted=0.702550, expected=0.690000
predicted=0.692550, expected=0.690000
predicted=0.692550, expected=0.760000
predicted=0.762550, expected=0.790000
predicted=0.792550, expected=0.780000
predicted=0.782550, expected=0.800000
predicted=0.802550, expected=0.680000
predicted=0.682550, expected=0.690000
predicted=0.692550, expected=0.730000
predicted=0.732550, expected=0.850000
predicted=0.852550, expected=0.880000
predicted=0.882550, expected=0.780000
predicted=0.782550, expected=0.800000
predicted=0.802550, expected=0.820000
predicted=0.822550, expected=0.780000
predicted=0.782550, expected=0.790000
predicted=0.792550, expected=0.790000
predicted=0.792550, expected=0.760000
predicted=0.762550, expected=0.780000
predicted=0.782550, expected=0.790000
predicted=0.792550, expected=0.790000
predicted=0.792550, expected=0.750000
predicted=0.752550, expected=0.760000
predicted=0.762550, expected=0.830000
predicted=0.832550, expected=0.860000
predicted=0.862550, expected=0.840000
predicted=0.842550, expected=0.880000
predicted=0.882550, expected=0.860000
predicted=0.862550, expected=0.830000
predicted=0.832550, expected=0.870000
predicted=0.872550, expected=0.840000
predicted=0.842550, expected=0.820000
predicted=0.822550, expected=0.810000
predicted=0.812550, expected=0.840000
In [ ]:
pred_inv = pd.DataFrame(predictions_diff, columns = ['pred_diff'])
pred_inv.index = pd.date_range(start="2019-08-16",end="2020-12-15" )
merge_inv = pd.merge(pred_inv, merge, how='inner', left_index=True, right_index=True)
merge_inv.tail()
Out[ ]:
pred_diff pred test_yieldsp
2020-12-11 0.83255 0.827924 0.87
2020-12-12 0.87255 0.875987 0.84
2020-12-13 0.84255 0.841573 0.82
2020-12-14 0.82255 0.818819 0.81
2020-12-15 0.81255 0.814550 0.84
In [ ]:
# calculate RMSE
rmse = sqrt(mean_squared_error(merge_inv['test_yieldsp'], merge_inv['pred_diff']))
print('Test RMSE from the differenced model - ARIMA(3,1,3): %.5f' % rmse)
Test RMSE from the differenced model - ARIMA(3,1,3): 0.05147

Thus, the final RMSE for the ARMA(1,0,3) model is 0.05147.

ARIMA(1,0,3) vs ARIMA(3,1,3)

Descriptive statistics of the residuals from both ARIMA models

Residuals from ARIMA(1, 0, 3) Residuals from Difference ARIMA(3, 1, 3)
mean -0.000066 0.000555
std 0.077213 0.077457
min -1.214803 -1.136089
25% -0.037550 -0.037421
50% -0.001489 -0.000781
75% 0.036286 0.037110
max 0.753097 0.750472

Plotting the ARIMA(1,0,3) model on yield spread vs ARIMA(3,1,3) model on the differenced yield spread

In [ ]:
# line plot shows that compares the actual yieldsp with rolling forecast predictions from leveled and diff ARIMA
fig_arima_diff = merge_inv.iplot(asFigure=True, kind='scatter', xTitle='Date', 
                                 yTitle='Predicted and Actual Yield Spread')
fig_arima_diff.update_layout(
    title={
        'text': "Actual and Forecasted Yield Spread from ARIMA(1,0,3) and ARIMA(3,1,3)",
        'y':0.9,
        'x':0.5,
        'xanchor': 'center',
        'yanchor': 'top'})

fig_arima_diff.show()

ARCH and GARCH Models

The author uses (G)ARCH models that treat heteroskedasticity by modeling the variance.

Empirically, after constructing ARIMA, we can use (G)ARCH to model the expected variance on the residuals, 
provided that the residuals are not serially correlated and don't have any trend and seasonal patterns.

When fitting an AR(p), we observe the decay of the $p$ lag displayed in the ACF plot of the residuals and the squared residuals. To apply ARCH we need to ensure that the mean of the residuals = 0.

In [ ]:
resid_arima['resid_arima_squared'] = resid_arima['resid_arima']**2 
resid_arima.head()
Out[ ]:
resid_arima resid_arima_squared
0 0.548815 0.301198
1 -0.078649 0.006186
2 0.194757 0.037930
3 0.020966 0.000440
4 0.039273 0.001542

Checking stationarity for resid_arima and resid_arima_squared

In [ ]:
for variable in resid_arima.columns:
    test_stationarity_acf_pacf(resid_arima[variable], sample=0.20, maxlag=30)                   
  • Whilst the residuals from ARIMA(1,0,3) appear to be white noise process, the squared residuals are highly autocorrelated.

  • The slow decay of successive lags indicate conditional heteroskedasticity.

Fitting an ARCH(1,0,3) model

Given that the lags in both the PACF and ACF plots are significant, the model would be better fit with both AR and MA parameters. So, we have fit a GARCH(1,3) model and checked how the residuals from GARCH(1,3) behave.

In [ ]:
# Now we can fit the arch model using the best fit arima model parameters
ar = arch_model(resid_arima['resid_arima'], p=1, o=0, q=3, power=1.0)
res = ar.fit(update_freq=5)
print(res.summary())
Iteration:      5,   Func. Count:     62,   Neg. LLF: -12300.528985841454
Iteration:     10,   Func. Count:    107,   Neg. LLF: -12453.98029286434
Iteration:     15,   Func. Count:    148,   Neg. LLF: -12467.454046611692
Optimization terminated successfully.    (Exit mode 0)
            Current function value: -12467.459248161376
            Iterations: 18
            Function evaluations: 164
            Gradient evaluations: 17
                    Constant Mean - AVGARCH Model Results                     
==============================================================================
Dep. Variable:            resid_arima   R-squared:                      -0.001
Mean Model:             Constant Mean   Adj. R-squared:                 -0.001
Vol Model:                    AVGARCH   Log-Likelihood:                12467.5
Distribution:                  Normal   AIC:                          -24922.9
Method:            Maximum Likelihood   BIC:                          -24880.1
                                        No. Observations:                 9253
Date:                Sat, Dec 26 2020   Df Residuals:                     9247
Time:                        04:52:08   Df Model:                            6
                                   Mean Model                                  
===============================================================================
                  coef    std err          t      P>|t|        95.0% Conf. Int.
-------------------------------------------------------------------------------
mu         -1.9980e-03  6.074e-04     -3.289  1.004e-03 [-3.188e-03,-8.075e-04]
                              Volatility Model                              
============================================================================
                 coef    std err          t      P>|t|      95.0% Conf. Int.
----------------------------------------------------------------------------
omega      1.7556e-03  3.811e-04      4.606  4.096e-06 [1.009e-03,2.503e-03]
alpha[1]       0.1142  1.362e-02      8.380  5.276e-17   [8.747e-02,  0.141]
beta[1]        0.8000      0.333      2.400  1.638e-02     [  0.147,  1.453]
beta[2]    7.4291e-12      0.602  1.234e-11      1.000     [ -1.180,  1.180]
beta[3]        0.0859      0.283      0.304      0.761     [ -0.468,  0.640]
============================================================================

Covariance estimator: robust

We can see that the true parameters: $\Omega, \alpha, \beta_1, \beta_2, \beta_3$, fall within the respective confidence intervals.

Thus, we can write GARCH(1, 3) as:

$$ \sigma_t^2 = 1.7556*10^{-03} + 0.1142 \sigma_{t-1}^2 + 0.8 \epsilon_{t-1}^2 + 7.4291*10^{-12} \epsilon_{t-2}^2 + 0.0859 \epsilon_{t-3}^2 $$
In [ ]:
resid_garch = pd.DataFrame(res.resid).rename(columns = {'resid': 'resid_garch'})
resid_garch['resid_garch_sq'] = resid_garch['resid_garch']**2
resid_garch.head()
Out[ ]:
resid_garch resid_garch_sq
0 0.550813 0.303395
1 -0.076651 0.005875
2 0.196755 0.038713
3 0.022964 0.000527
4 0.041271 0.001703

Stationarity analysis of resid_garch and resid_garch_squared

In [ ]:
for variable in resid_garch.columns:
    test_stationarity_acf_pacf(resid_garch[variable], sample=0.20, maxlag=30) 
  • The residuls (from GARCH) plot looks like a realisation of a discrete white noise process, indicating a good fit.
  • But, the squared residuals are still not white noise.

So, GARCH(1, 3) has not "explained" the serial correlation present in the squared residuals, inhibiting us from predicting in the test set.

Multivariate forecasting

Gathering the exogenous variables

In [ ]:
def parser(x):
    return datetime.strptime('190'+x, '%Y-%m')

week = ["FRED/CCSA", # UI claims - weekly;
    
        "FRED/WALCL", # Total Assets (Less Eliminations From Consolidation) - weekly;
    
        "FRED/RPONTSYD"] # Overnight Repurchase Agreements: Treasury Securities Purchased by Fed in OMO - weekly
    
In [ ]:
daily = [
    
    "FRED/T10Y3M", # 10 year treasury constant maturity - 3 month tres cons mat
    
    "FRED/TEDRATE",  # TED Spread - daily ;
    
    "FRED/THREEFF1" , # Fitted Instantaneous Forward Rate 1 Year Hence - daily;
    
    "FRED/USRECD", # NBER based recession indicator - daily
    
    "FRED/T1YFF" , # 1-Year Treasury Constant Maturity Minus Federal Funds Rate
    
    "FRED/THREEFYTP10" ] # Term Premium on a 10 Year Zero Coupon Bond - daily

yieldd = quandl.get(daily, start_date = "1990-01-01", end_date="2020-10-01", collapse = 'daily',date_parser=parser)
yieldd.columns = [
                  'yieldsp',
                  'ted', 
                  'forward1yr',
                  'rec_ind',
                  '1yr-ffr',
                  'termpr'
                  ]
   
yieldd = yieldd.fillna(method = 'bfill')
yieldd.tail()
Out[ ]:
yieldsp ted forward1yr rec_ind 1yr-ffr termpr
Date
2020-09-27 0.56 0.11 0.1757 1.0 0.03 -0.8797
2020-09-28 0.56 0.11 0.1757 1.0 0.03 -0.8797
2020-09-29 0.57 0.14 0.1713 1.0 0.03 -0.8838
2020-09-30 0.59 0.13 0.1811 1.0 0.03 -0.8731
2020-10-01 0.59 0.14 0.1811 1.0 0.03 -0.8710
In [ ]:
fredf = {}
fredf['sahm'] = fred.get_series('SAHMREALTIME', observation_start='1990-01-01', observation_end='2020-10-01')
fredf['infexp'] = fred.get_series('MICH', observation_start='1990-01-01', observation_end='2020-10-01')
fredf['vix'] = fred.get_series('VIXCLS', observation_start='1990-01-01', observation_end='2020-10-01')
fredf = pd.DataFrame(fredf)

# apply cubic spline interpolation to get daily value of sahm rule from monthly values
# bfill = use next valid observation to fill gap.
upsample = fredf.resample('D').interpolate(method='cubic').fillna(method = 'bfill')
upsample.index.name = 'Date'
upsample.tail()
Out[ ]:
sahm infexp vix
Date
2020-09-27 4.370888 2.519419 25.691582
2020-09-28 4.329146 2.536514 26.190000
2020-09-29 4.286764 2.555607 26.270000
2020-09-30 4.243722 2.576751 26.370000
2020-10-01 4.200000 2.600000 26.700000
In [ ]:
yield_d = pd.merge(yieldd, upsample, on = 'Date')
yield_d.head()
Out[ ]:
yieldsp ted forward1yr rec_ind 1yr-ffr termpr sahm infexp vix
Date
1990-01-01 0.11 0.80 7.8401 0.0 -0.73 1.6625 0.130000 4.100000 17.24
1990-01-02 0.11 0.80 7.8401 0.0 -0.73 1.6625 0.132076 4.121014 17.24
1990-01-03 0.10 0.75 7.8880 0.0 -0.52 1.6865 0.133962 4.140169 18.19
1990-01-04 0.14 0.79 7.8773 0.0 -0.47 1.6890 0.135664 4.157512 19.22
1990-01-05 0.20 0.84 7.8694 0.0 -0.41 1.7017 0.137187 4.173091 20.11
In [ ]:
yield_d.isnull().sum()
Out[ ]:
yieldsp       0
ted           0
forward1yr    0
rec_ind       0
1yr-ffr       0
termpr        0
sahm          0
infexp        0
vix           0
dtype: int64

D’Agostino’s $K^2$ Test

We use goodness-of-fit measure : the D’Agostino’s $K^2$ Test to check if the sample data on the yield spread arises from a Gaussian distributed population. Transforming the sample skewness and kurtosis derives the test statistic.

Test statistic:

Ho: data is normally distributed
Ha: data is kurtic and/or skewed (not normally distributed)
In [ ]:
from scipy import stats
stat, p = stats.normaltest(yield_d.yieldsp)
print('Statistics=%.3f, p=%.3f' % (stat, p))

alpha = 0.05
if p > alpha:
    print('Data looks Gaussian (fail to reject H0)')
else:
    print('Data does not look Gaussian (reject H0)')
Statistics=3205.902, p=0.000
Data does not look Gaussian (reject H0)

The small p-value of 0.00 rejects the null hypothesis. Hence, the sample of yield spread is not normally distributed.

In [ ]:
hist_data = [yield_d.yieldsp]
group_labels = ['distplot'] 

fig_kde = ff.create_distplot(hist_data, group_labels, bin_size=0.01)
fig_kde.update_layout(
    title={
        'text': "Histogram, KDE and a Rug Plot of the Yield Spread",
        'y':0.9,
        'x':0.5,
        'xanchor': 'center',
        'yanchor': 'top'})

fig_kde.show()
In [ ]:
# calculate kurtosis and skewness, to determine if the data distribution departs from the normal distribution.
print( 'Kurtosis of normal distribution: {}'.format(stats.kurtosis(yield_d.yieldsp)))
print( 'Skewness of normal distribution: {}'.format(stats.skew(yield_d.yieldsp)))
Kurtosis of normal distribution: -1.039932153725899
Skewness of normal distribution: -0.056269426368131585

The yield spread data is moderately skewed as its skewness is -0.0562, and has heavy tails (very kurtotic) becasue the kurtosis is -1.0399.

Exploring the Stationarity of the Variables

Augmented Dickey Fuller test of the variables

In [ ]:
def adfuller_test(series, signif = 0.05, name = '', verbose = False):
    """
        Perform ADFuller to test for stationarity of a given series and print report
    """
    
    r = adfuller(series, autolag = 'AIC')
    output = {'test_statistic': round(r[0], 4), 'pvalue': round(r[1], 4), 'n_lags':round(r[2], 4), 'n_obs':r[3]}
    p_value = output['pvalue']

    def adjust(val, length = 6): 
        return str(val).ljust(length)
    
    print(f'     Augmented Dickey-Fuller Test on "{name}"', "\n    ", '-'*47)
    print(f'  Null Hypothesis: Data has unit too. Non-Stationary.')
    print(f'  Significance Level    = {signif}')
    print(f'  Test Statistic        = {output["test_statistic"]}')
    print(f'  No. of Lags Chosen    = {output["n_lags"]}')
          
    for key, val in r[4].items():
          print(f'  Critical Value {adjust(key)} = {round(val, 3)}')
          
    if p_value <= signif:
          print(f" => P-Value = {p_value}. Rejecting null hypothesis.")
          print(f" => Series is stationary.")
    else:
          print(f" => P-Value = {p_value}. Weak evidence to reject the Null Hypothesis.")
          print(f" => Series is non-stationary.")
 
In [ ]:
data_list = yield_d[list(yield_d.columns)]

for name, column in data_list.iteritems():
          adfuller_test(column, name = column.name)
          print('\n')
     Augmented Dickey-Fuller Test on "yieldsp" 
     -----------------------------------------------
  Null Hypothesis: Data has unit too. Non-Stationary.
  Significance Level    = 0.05
  Test Statistic        = -2.4551
  No. of Lags Chosen    = 30
  Critical Value 1%     = -3.431
  Critical Value 5%     = -2.862
  Critical Value 10%    = -2.567
 => P-Value = 0.1268. Weak evidence to reject the Null Hypothesis.
 => Series is non-stationary.


     Augmented Dickey-Fuller Test on "ted" 
     -----------------------------------------------
  Null Hypothesis: Data has unit too. Non-Stationary.
  Significance Level    = 0.05
  Test Statistic        = -5.1091
  No. of Lags Chosen    = 39
  Critical Value 1%     = -3.431
  Critical Value 5%     = -2.862
  Critical Value 10%    = -2.567
 => P-Value = 0.0. Rejecting null hypothesis.
 => Series is stationary.


     Augmented Dickey-Fuller Test on "forward1yr" 
     -----------------------------------------------
  Null Hypothesis: Data has unit too. Non-Stationary.
  Significance Level    = 0.05
  Test Statistic        = -1.5291
  No. of Lags Chosen    = 3
  Critical Value 1%     = -3.431
  Critical Value 5%     = -2.862
  Critical Value 10%    = -2.567
 => P-Value = 0.5191. Weak evidence to reject the Null Hypothesis.
 => Series is non-stationary.


     Augmented Dickey-Fuller Test on "rec_ind" 
     -----------------------------------------------
  Null Hypothesis: Data has unit too. Non-Stationary.
  Significance Level    = 0.05
  Test Statistic        = -3.7435
  No. of Lags Chosen    = 0
  Critical Value 1%     = -3.431
  Critical Value 5%     = -2.862
  Critical Value 10%    = -2.567
 => P-Value = 0.0035. Rejecting null hypothesis.
 => Series is stationary.


     Augmented Dickey-Fuller Test on "1yr-ffr" 
     -----------------------------------------------
  Null Hypothesis: Data has unit too. Non-Stationary.
  Significance Level    = 0.05
  Test Statistic        = -4.9412
  No. of Lags Chosen    = 37
  Critical Value 1%     = -3.431
  Critical Value 5%     = -2.862
  Critical Value 10%    = -2.567
 => P-Value = 0.0. Rejecting null hypothesis.
 => Series is stationary.


     Augmented Dickey-Fuller Test on "termpr" 
     -----------------------------------------------
  Null Hypothesis: Data has unit too. Non-Stationary.
  Significance Level    = 0.05
  Test Statistic        = -1.3734
  No. of Lags Chosen    = 3
  Critical Value 1%     = -3.431
  Critical Value 5%     = -2.862
  Critical Value 10%    = -2.567
 => P-Value = 0.595. Weak evidence to reject the Null Hypothesis.
 => Series is non-stationary.


     Augmented Dickey-Fuller Test on "sahm" 
     -----------------------------------------------
  Null Hypothesis: Data has unit too. Non-Stationary.
  Significance Level    = 0.05
  Test Statistic        = -5.5135
  No. of Lags Chosen    = 38
  Critical Value 1%     = -3.431
  Critical Value 5%     = -2.862
  Critical Value 10%    = -2.567
 => P-Value = 0.0. Rejecting null hypothesis.
 => Series is stationary.


     Augmented Dickey-Fuller Test on "infexp" 
     -----------------------------------------------
  Null Hypothesis: Data has unit too. Non-Stationary.
  Significance Level    = 0.05
  Test Statistic        = -6.3951
  No. of Lags Chosen    = 40
  Critical Value 1%     = -3.431
  Critical Value 5%     = -2.862
  Critical Value 10%    = -2.567
 => P-Value = 0.0. Rejecting null hypothesis.
 => Series is stationary.


     Augmented Dickey-Fuller Test on "vix" 
     -----------------------------------------------
  Null Hypothesis: Data has unit too. Non-Stationary.
  Significance Level    = 0.05
  Test Statistic        = -5.8964
  No. of Lags Chosen    = 33
  Critical Value 1%     = -3.431
  Critical Value 5%     = -2.862
  Critical Value 10%    = -2.567
 => P-Value = 0.0. Rejecting null hypothesis.
 => Series is stationary.


Variable p -Value Stationary?
yieldsp 0.1268 No
ted 0.000 Yes
forward1yr 0.5191 No
rec_ind 0.0035 Yes
1yr-ffr 0.000 Yes
termpr 0.595 No
sahm 0.000 Yes
infexp 0.000 Yes
vix 0.000 Yes

Here, I get results different from the author for sahm.

Stationarizing variables by taking first diff

In [ ]:
# stationarizes the variables by taking the first difference, and concats them with the dataset
yield_d1 = pd.concat([yield_d, yield_d[['forward1yr','termpr','yieldsp']]
                      .diff().add_suffix("_diff")], axis = 1)

# drops rows with NaN values, and replaces inf values with 0
stat1 = yield_d1.dropna(inplace = False).replace([np.inf, -np.inf], 0).dropna(axis=1)

AD Fuller test on the differenced variables

I have done AD Fuller test on the differenced variables once again to verify that they are stationary.

In [ ]:
stat_list1 = stat1[['forward1yr_diff', 'termpr_diff','yieldsp_diff']]

# ADF Test on the first differenced values each column
for name, column in stat_list1.iteritems():
          adfuller_test(column, name = column.name)
          print('\n')
     Augmented Dickey-Fuller Test on "forward1yr_diff" 
     -----------------------------------------------
  Null Hypothesis: Data has unit too. Non-Stationary.
  Significance Level    = 0.05
  Test Statistic        = -54.7771
  No. of Lags Chosen    = 2
  Critical Value 1%     = -3.431
  Critical Value 5%     = -2.862
  Critical Value 10%    = -2.567
 => P-Value = 0.0. Rejecting null hypothesis.
 => Series is stationary.


     Augmented Dickey-Fuller Test on "termpr_diff" 
     -----------------------------------------------
  Null Hypothesis: Data has unit too. Non-Stationary.
  Significance Level    = 0.05
  Test Statistic        = -54.4446
  No. of Lags Chosen    = 2
  Critical Value 1%     = -3.431
  Critical Value 5%     = -2.862
  Critical Value 10%    = -2.567
 => P-Value = 0.0. Rejecting null hypothesis.
 => Series is stationary.


     Augmented Dickey-Fuller Test on "yieldsp_diff" 
     -----------------------------------------------
  Null Hypothesis: Data has unit too. Non-Stationary.
  Significance Level    = 0.05
  Test Statistic        = -18.5651
  No. of Lags Chosen    = 30
  Critical Value 1%     = -3.431
  Critical Value 5%     = -2.862
  Critical Value 10%    = -2.567
 => P-Value = 0.0. Rejecting null hypothesis.
 => Series is stationary.


Thus, we have the following:

Stationary Variables ted, rec_ind, 1yr-ffr, vix, infexp, sahm
Variables that need first differencing forward1yr, termpr, yieldsp

ARIMA with Exogenous variables : SARIMAX(2,1,2)

The author has modeled SARIMAX(2,1,2) without any seasonal component and incorporated the stationary exogenous variables.

In [ ]:
train, test = np.split(stat1, [int(.95 *len(stat1))])

exog_train = train[['ted', 'rec_ind', '1yr-ffr', 'vix', 'termpr_diff', 'forward1yr_diff', 'infexp']]
exog_test = test[['ted', 'rec_ind', '1yr-ffr', 'vix', 'termpr_diff','forward1yr_diff', 'infexp']]
In [ ]:
exog = stat1[['ted', 'rec_ind', '1yr-ffr', 'vix', 'termpr', 'forward1yr', 'infexp', 'sahm', '1yr-ffr']]
exog_test_sarima = test[['ted', 'rec_ind', '1yr-ffr', 'vix', 'termpr', 'forward1yr', 'infexp', 'sahm', '1yr-ffr']]

mod = sm.tsa.statespace.SARIMAX(stat1['yieldsp'], exog = exog,
                                order = (2, 1, 2),
                                seasonal_order = (0, 0, 0, 0),
                                enforce_stationarity = True,
                                enforce_invertibility = True)

results = mod.fit()

print(results.summary())
                           Statespace Model Results                           
==============================================================================
Dep. Variable:                yieldsp   No. Observations:                11231
Model:               SARIMAX(2, 1, 2)   Log Likelihood               24898.307
Date:                Sat, 26 Dec 2020   AIC                         -49768.613
Time:                        04:53:32   BIC                         -49666.045
Sample:                    01-02-1990   HQIC                        -49734.094
                         - 10-01-2020                                         
Covariance Type:                  opg                                         
==============================================================================
                 coef    std err          z      P>|z|      [0.025      0.975]
------------------------------------------------------------------------------
ted            0.4031      0.002    248.608      0.000       0.400       0.406
rec_ind       -0.0072      0.020     -0.367      0.714      -0.046       0.031
1yr-ffr        0.0065      0.000     16.714      0.000       0.006       0.007
vix           -0.0002      0.000     -1.244      0.213      -0.000    9.08e-05
termpr         2.4013      0.010    240.108      0.000       2.382       2.421
forward1yr    -0.4190      0.005    -77.022      0.000      -0.430      -0.408
infexp        -0.0168      0.018     -0.953      0.341      -0.051       0.018
sahm           0.0259      0.016      1.662      0.097      -0.005       0.056
1yr-ffr        0.0065      0.000     16.714      0.000       0.006       0.007
ar.L1          1.0625      0.019     55.446      0.000       1.025       1.100
ar.L2         -0.6652      0.018    -37.524      0.000      -0.700      -0.630
ma.L1         -0.9914      0.021    -47.198      0.000      -1.033      -0.950
ma.L2          0.5403      0.020     27.323      0.000       0.502       0.579
sigma2         0.0007   3.27e-06    212.366      0.000       0.001       0.001
===================================================================================
Ljung-Box (Q):                      477.68   Jarque-Bera (JB):            162832.82
Prob(Q):                              0.00   Prob(JB):                         0.00
Heteroskedasticity (H):               0.30   Skew:                            -0.57
Prob(H) (two-sided):                  0.00   Kurtosis:                        21.62
===================================================================================

Warnings:
[1] Covariance matrix calculated using the outer product of gradients (complex-step).
[2] Covariance matrix is singular or near-singular, with condition number 1.27e+17. Standard errors may be unstable.

The SARIMAX(2,1,2) model is:

$$ yieldsp_t = 0.4031 ted_t -0.0072 \text{rec_ind}_t + 0.0065 \text{1yrffr}_t - 0.0002 vix_t + 2.4013 \Delta termpr_t - 0.4190 \Delta forward1yr_t - 0.0168 \text{infexp}_t + 0.0259 sahm_t + 0.0065 \text{1yrffr}_t + 1.0625 yieldsp_{t-1} - 0.6652 yieldsp_{t-2} - 0.9914 \epsilon_{t-1} + 0.5403 \epsilon_{t-2}$$

with an AIC of -49768.613

Model Diagnostics

In [ ]:
# run model diagnostics to ensure that none of the assumptions made by the model have been violated.
results.plot_diagnostics(figsize=(20, 12))
plt.show()

Forecasting using SARIMAX(2,1,2)

In [ ]:
pred_dynamic = results.get_prediction(start = pd.to_datetime('2018-11-24'), dynamic = True, 
                                      exog = exog_test_sarima, full_results = True)
                                      
pred_dynamic_ci = pred_dynamic.conf_int(exog = exog_test)
In [ ]:
# plot the real and forecasted values of the yield spread 

ax = stat1['yieldsp']['2018-10-28':].plot(label='observed', figsize=(20, 10))
pred_dynamic.predicted_mean.plot(label='Dynamic Forecast', ax = ax, fontsize = 16)

ax.fill_between(pred_dynamic_ci.index,
                pred_dynamic_ci.iloc[:, 0],
                pred_dynamic_ci.iloc[:, 1], color='k', alpha=.25)

ax.fill_betweenx(ax.get_ylim(), pd.to_datetime('2018-11-01'), stat1['yieldsp'].index[-1],
                 alpha=.1, zorder=-1)

ax.set_xlabel('Date', fontsize = 18)
ax.set_ylabel('Yield Spread', fontsize = 18)
ax.set_title("Actual and Predicted Yield Spread from SARIMAX(2,1,2)", fontsize = 20)

plt.legend()
plt.show()
In [ ]:
y_forecasted = pred_dynamic.predicted_mean
y_truth = stat1['yieldsp']['2018-11-24':]

# Compute the mean square error
rmse = sqrt(((y_forecasted - y_truth) ** 2).mean())
print('The Root Mean Squared Error of our forecasts is {}'.format(round(rmse, 4)))
The Root Mean Squared Error of our forecasts is 0.4744
In [ ]:
exog_test.tail()
Out[ ]:
ted rec_ind 1yr-ffr vix termpr_diff forward1yr_diff infexp
Date
2020-09-27 0.11 1.0 0.03 25.691582 0.0000 0.0000 2.519419
2020-09-28 0.11 1.0 0.03 26.190000 0.0000 0.0000 2.536514
2020-09-29 0.14 1.0 0.03 26.270000 -0.0041 -0.0044 2.555607
2020-09-30 0.13 1.0 0.03 26.370000 0.0107 0.0098 2.576751
2020-10-01 0.14 1.0 0.03 26.700000 0.0021 0.0000 2.600000

SARIMAX(2,1,2) predicts rec_ind 1.0 during September 2020 which is as expected.

Granger Causality Test

The author has appliead the Granger Causality test to determine variables have causal relationship with $\Delta yieldsp$ .

Specifying the maximum number of lags specified as 40, the equation to test the null hypothesis is:

$$ \Delta yieldsp_t = \alpha_0 + \sum_{i=1}^{40} \alpha_i \ yieldsp_{t-i} + \sum_{j=1}^{40} \beta_j \ x_{t-j} $$

where $x_j$ refers to a stationary explanatory variable.

In [ ]:
# granger causality for all variables in the train set
stat_train = train[['termpr_diff', 'forward1yr_diff', 'ted', 'vix', 'yieldsp', 'yieldsp_diff',
                    'rec_ind', 'infexp', '1yr-ffr']]

maxlag=40

def grangers_causality_matrix(data, variables, test = 'ssr_chi2test', verbose=False):

    dataset = pd.DataFrame(np.zeros((len(variables), len(variables))), columns=variables, index=variables)

    for c in dataset.columns:
        for r in dataset.index:
            test_result = grangercausalitytests(data[[r,c]], maxlag=maxlag, verbose=False)
            p_values = [round(test_result[i+1][0][test][1],4) for i in range(maxlag)]
            if verbose: print(f'Y = {r}, X = {c}, P Values = {p_values}')

            min_p_value = np.min(p_values)
            dataset.loc[r,c] = min_p_value

    dataset.columns = [var + '_x' for var in variables]

    dataset.index = [var + '_y' for var in variables]

    return dataset

granger_matrix = grangers_causality_matrix(stat_train, variables = stat_train.columns) 
In [ ]:
plt.figure(figsize = (10,8))
sn.heatmap(granger_matrix, annot=True, cmap="YlGnBu")
Out[ ]:
<matplotlib.axes._subplots.AxesSubplot at 0x7f8b83d9c5c0>

The row are the response ($y$) and the columns are the predictor series ($x$).

All the variables except infexp granger cause yieldsp as each of their p-vaalues = 0.00 < 0.05.

Vector Autoregression (VAR)

Determining the optimal lag order

The author has used the AIC criteria to find the optimal lag period for the VAR model.

In [ ]:
# should use only the stationary variables for VAR
stat_train = train[['yieldsp_diff','termpr_diff', 'forward1yr_diff', 'ted', 'vix', 'rec_ind', '1yr-ffr']]

from statsmodels.tsa.api import VAR

AIC = {}
best_aic, best_order = np.inf, 0

for i in range(1,50):
    model = VAR(endog=stat_train.values)
    model_result = model.fit(maxlags=i)
    AIC[i] = model_result.aic
    
    if AIC[i] < best_aic:
        best_aic = AIC[i]
        best_order = i
        
print('Best Order', best_order, 'BEST AIC:', best_aic)
Best Order 45 BEST AIC: -40.86781412110635
In [ ]:
plt.figure(figsize=(20,6))
plt.plot(range(len(AIC)), list(AIC.values()))
plt.plot([best_order-1], [best_aic], marker='o', markersize=8, color="blue")
plt.xticks(range(len(AIC)), range(1,50))
plt.xlabel('lags'); plt.ylabel('AIC')
plt.title("Measure of VAR's Performance via AIC over Lags and the best AIC")
np.set_printoptions(False)

Building a VAR(45) model

We have aggregated the variables used in VAR(45) as a (7 × 1) vector of series variables:

$$ Y_t = [\Delta yieldsp_t \ \ \Delta termpr_t \ \ \Delta forward1yr_t \ \ 1yrffr_t \ \ \text{rec_ind}_t \ \ ted_t \ \ vix_t]' $$

VAR(45) is a seemingly unrelated regression (SUR) model with $k = 7$ deterministic regressors and $l = 45$ lagged variables of the form:

$$ \Delta yieldsp_{t} = \alpha_1 + \sum_{i=1}^{7} \sum_{l=1}^{45} \beta_{i,l}^{\Delta yieldsp} \Delta yieldsp_{t-1} + \epsilon_{\Delta yieldsp_{t}} $$$$ \Delta termpr_{t} = \alpha_2 + \sum_{i=1}^{7} \sum_{l=1}^{45} \beta_{i,l}^{\Delta termpr} \Delta yieldsp_{t-1} + \epsilon_{\Delta termpr_{t}} $$$$ \Delta forward1yr_{t} = \alpha_3 + \sum_{i=1}^{7} \sum_{l=1}^{45} \beta_{i,l}^{\Delta forward1yr} \Delta yieldsp_{t-1} + \epsilon_{\Delta forward1yr_{t}} $$$$ \text{1yr-frr}_{t} = \alpha_4 + \sum_{i=1}^{7} \sum_{l=1}^{45} \beta_{i,l}^{\text{1yr-frr}} \Delta yieldsp_{t-1} + \epsilon_{\text{1yr-frr}_t} $$$$ \text{rec_ind}_{t} = \alpha_5 + \sum_{i=1}^{7} \sum_{l=1}^{45} \beta_{i,l}^{\text{rec_ind}} \Delta yieldsp_{t-1} + \epsilon_{\text{rec_ind}_t} $$$$ ted_{t} = \alpha_6 + \sum_{i=1}^{7} \sum_{l=1}^{45} \beta_{i,l}^{ted} \Delta yieldsp_{t-1} + \epsilon_{ted_{t}} $$$$ vix_{t} = \alpha_7 + \sum_{i=1}^{7} \sum_{l=1}^{45} \beta_{i,l}^{vix} \Delta yieldsp_{t-1} + \epsilon_{vix_{t}} $$

where $ \epsilon_{t} = [ \epsilon_{{\Delta yieldsp}_{t}} \ \ \epsilon_{{\Delta termpr}_{t}} \ \ \epsilon_{{\Delta forward1yr}_{t}} \ \ \epsilon_{{1yrffr}_{t}} \ \ \epsilon_{{\text{rec_ind}}_{t}} \ \ \epsilon_{{ted}_{t}} \ \ \epsilon_{{vix}_{t}}]' $ is a white noise process of "innovations".

$ \epsilon_{t} = [ \epsilon_{{\Delta yieldsp}_{t}} \ \ \epsilon_{{\Delta termpr}_{t}} \ \ \epsilon_{{\Delta forward1yr}_{t}} \ \ \epsilon_{{1yrffr}_{t}} \ \ \epsilon_{{\text{rec_ind}}_{t}} \ \ \epsilon_{{ted}_{t}} \ \ \epsilon_{{vix}_{t}}]' $

In [ ]:
# fit the VAR model 
mod = VAR(stat_train)
res = mod.fit(best_order) 
res.summary() 
Out[ ]:
  Summary of Regression Results   
==================================
Model:                         VAR
Method:                        OLS
Date:           Sat, 26, Dec, 2020
Time:                     04:57:58
--------------------------------------------------------------------
No. of Equations:         7.00000    BIC:                   -39.3540
Nobs:                     10624.0    HQIC:                  -40.3569
Log likelihood:           113778.    FPE:                1.78397e-18
AIC:                     -40.8678    Det(Omega_mle):     1.45306e-18
--------------------------------------------------------------------
Results for equation yieldsp_diff
======================================================================================
                         coefficient       std. error           t-stat            prob
--------------------------------------------------------------------------------------
const                       0.003419         0.001659            2.061           0.039
L1.yieldsp_diff             0.089249         0.022530            3.961           0.000
L1.termpr_diff             -0.026905         0.078855           -0.341           0.733
L1.forward1yr_diff         -0.026057         0.033124           -0.787           0.431
L1.ted                      0.015610         0.016844            0.927           0.354
L1.vix                      0.001006         0.000461            2.180           0.029
L1.rec_ind                  0.023949         0.021694            1.104           0.270
L1.1yr-ffr                 -0.000297         0.003780           -0.079           0.937
L2.yieldsp_diff            -0.008111         0.022629           -0.358           0.720
L2.termpr_diff              0.031829         0.083419            0.382           0.703
L2.forward1yr_diff         -0.025241         0.036680           -0.688           0.491
L2.ted                     -0.078765         0.023778           -3.313           0.001
L2.vix                     -0.000762         0.000741           -1.028           0.304
L2.rec_ind                  0.008887         0.030644            0.290           0.772
L2.1yr-ffr                  0.003564         0.004423            0.806           0.420
L3.yieldsp_diff            -0.100189         0.022700           -4.414           0.000
L3.termpr_diff              0.265635         0.083757            3.172           0.002
L3.forward1yr_diff         -0.048866         0.036891           -1.325           0.185
L3.ted                      0.058880         0.023771            2.477           0.013
L3.vix                      0.000910         0.000781            1.165           0.244
L3.rec_ind                  0.028149         0.030670            0.918           0.359
L3.1yr-ffr                 -0.007339         0.004423           -1.659           0.097
L4.yieldsp_diff            -0.094357         0.022971           -4.108           0.000
L4.termpr_diff              0.267034         0.083961            3.180           0.001
L4.forward1yr_diff         -0.103877         0.037035           -2.805           0.005
L4.ted                      0.016352         0.023773            0.688           0.492
L4.vix                      0.000371         0.000788            0.471           0.638
L4.rec_ind                 -0.100271         0.030680           -3.268           0.001
L4.1yr-ffr                 -0.002456         0.004415           -0.556           0.578
L5.yieldsp_diff            -0.116204         0.023060           -5.039           0.000
L5.termpr_diff              0.129609         0.084061            1.542           0.123
L5.forward1yr_diff          0.020187         0.037078            0.544           0.586
L5.ted                      0.011440         0.023806            0.481           0.631
L5.vix                     -0.001367         0.000787           -1.737           0.082
L5.rec_ind                  0.027126         0.030698            0.884           0.377
L5.1yr-ffr                  0.012367         0.004415            2.801           0.005
L6.yieldsp_diff            -0.052948         0.023150           -2.287           0.022
L6.termpr_diff             -0.001967         0.084079           -0.023           0.981
L6.forward1yr_diff          0.004336         0.037078            0.117           0.907
L6.ted                     -0.008789         0.023821           -0.369           0.712
L6.vix                      0.001027         0.000786            1.306           0.191
L6.rec_ind                 -0.016528         0.030689           -0.539           0.590
L6.1yr-ffr                 -0.012483         0.004417           -2.826           0.005
L7.yieldsp_diff             0.012741         0.023164            0.550           0.582
L7.termpr_diff             -0.083763         0.084082           -0.996           0.319
L7.forward1yr_diff          0.013047         0.037086            0.352           0.725
L7.ted                     -0.005919         0.023802           -0.249           0.804
L7.vix                     -0.000327         0.000785           -0.417           0.677
L7.rec_ind                  0.058420         0.030693            1.903           0.057
L7.1yr-ffr                  0.004629         0.004425            1.046           0.296
L8.yieldsp_diff            -0.024221         0.023198           -1.044           0.296
L8.termpr_diff              0.044974         0.084276            0.534           0.594
L8.forward1yr_diff         -0.048557         0.037143           -1.307           0.191
L8.ted                     -0.039373         0.023818           -1.653           0.098
L8.vix                     -0.001197         0.000785           -1.525           0.127
L8.rec_ind                 -0.090930         0.030690           -2.963           0.003
L8.1yr-ffr                 -0.009689         0.004434           -2.185           0.029
L9.yieldsp_diff            -0.009042         0.023174           -0.390           0.696
L9.termpr_diff             -0.099272         0.084333           -1.177           0.239
L9.forward1yr_diff          0.083427         0.037172            2.244           0.025
L9.ted                      0.007391         0.023846            0.310           0.757
L9.vix                      0.001307         0.000785            1.666           0.096
L9.rec_ind                  0.112188         0.030707            3.654           0.000
L9.1yr-ffr                  0.006319         0.004432            1.426           0.154
L10.yieldsp_diff            0.013252         0.023182            0.572           0.568
L10.termpr_diff            -0.057132         0.084335           -0.677           0.498
L10.forward1yr_diff         0.006885         0.037185            0.185           0.853
L10.ted                     0.009979         0.023863            0.418           0.676
L10.vix                    -0.001059         0.000784           -1.350           0.177
L10.rec_ind                -0.069364         0.030735           -2.257           0.024
L10.1yr-ffr                 0.007739         0.004434            1.745           0.081
L11.yieldsp_diff           -0.048010         0.023163           -2.073           0.038
L11.termpr_diff             0.112460         0.084299            1.334           0.182
L11.forward1yr_diff        -0.032441         0.037205           -0.872           0.383
L11.ted                    -0.010963         0.023883           -0.459           0.646
L11.vix                    -0.000141         0.000784           -0.179           0.858
L11.rec_ind                 0.059691         0.030732            1.942           0.052
L11.1yr-ffr                -0.010828         0.004437           -2.440           0.015
L12.yieldsp_diff            0.021970         0.023169            0.948           0.343
L12.termpr_diff            -0.039116         0.084361           -0.464           0.643
L12.forward1yr_diff         0.003741         0.037219            0.101           0.920
L12.ted                    -0.035534         0.023895           -1.487           0.137
L12.vix                     0.000298         0.000783            0.380           0.704
L12.rec_ind                -0.041652         0.030730           -1.355           0.175
L12.1yr-ffr                 0.005124         0.004434            1.156           0.248
L13.yieldsp_diff           -0.061231         0.023165           -2.643           0.008
L13.termpr_diff             0.232113         0.084360            2.751           0.006
L13.forward1yr_diff        -0.076885         0.037224           -2.065           0.039
L13.ted                     0.068020         0.023904            2.846           0.004
L13.vix                     0.000156         0.000783            0.200           0.842
L13.rec_ind                -0.041686         0.030729           -1.357           0.175
L13.1yr-ffr                 0.000022         0.004430            0.005           0.996
L14.yieldsp_diff           -0.001819         0.023137           -0.079           0.937
L14.termpr_diff             0.128842         0.084278            1.529           0.126
L14.forward1yr_diff        -0.065932         0.037230           -1.771           0.077
L14.ted                    -0.016827         0.023921           -0.703           0.482
L14.vix                    -0.000060         0.000783           -0.077           0.939
L14.rec_ind                 0.004970         0.030699            0.162           0.871
L14.1yr-ffr                 0.009851         0.004430            2.224           0.026
L15.yieldsp_diff            0.015442         0.023147            0.667           0.505
L15.termpr_diff            -0.061863         0.084286           -0.734           0.463
L15.forward1yr_diff        -0.010102         0.037235           -0.271           0.786
L15.ted                    -0.029241         0.024031           -1.217           0.224
L15.vix                     0.000380         0.000785            0.484           0.629
L15.rec_ind                 0.067978         0.030668            2.217           0.027
L15.1yr-ffr                -0.009697         0.004469           -2.170           0.030
L16.yieldsp_diff           -0.045068         0.023138           -1.948           0.051
L16.termpr_diff             0.098802         0.084292            1.172           0.241
L16.forward1yr_diff        -0.015943         0.037230           -0.428           0.668
L16.ted                     0.040658         0.024056            1.690           0.091
L16.vix                    -0.000562         0.000787           -0.714           0.475
L16.rec_ind                -0.064091         0.030668           -2.090           0.037
L16.1yr-ffr                 0.005093         0.004476            1.138           0.255
L17.yieldsp_diff           -0.037595         0.023135           -1.625           0.104
L17.termpr_diff             0.060915         0.084289            0.723           0.470
L17.forward1yr_diff        -0.039120         0.037239           -1.050           0.293
L17.ted                    -0.020538         0.024062           -0.854           0.393
L17.vix                     0.000540         0.000786            0.687           0.492
L17.rec_ind                 0.117679         0.030665            3.838           0.000
L17.1yr-ffr                -0.000587         0.004471           -0.131           0.896
L18.yieldsp_diff           -0.035008         0.023153           -1.512           0.131
L18.termpr_diff             0.014250         0.084236            0.169           0.866
L18.forward1yr_diff         0.032546         0.037227            0.874           0.382
L18.ted                     0.021798         0.024058            0.906           0.365
L18.vix                    -0.000561         0.000786           -0.714           0.475
L18.rec_ind                 0.015753         0.030702            0.513           0.608
L18.1yr-ffr                -0.001307         0.004446           -0.294           0.769
L19.yieldsp_diff            0.021375         0.023160            0.923           0.356
L19.termpr_diff            -0.168812         0.084215           -2.005           0.045
L19.forward1yr_diff         0.063229         0.037219            1.699           0.089
L19.ted                    -0.015591         0.024086           -0.647           0.517
L19.vix                     0.000891         0.000785            1.135           0.257
L19.rec_ind                -0.100848         0.030732           -3.282           0.001
L19.1yr-ffr                 0.004955         0.004442            1.115           0.265
L20.yieldsp_diff           -0.041480         0.023145           -1.792           0.073
L20.termpr_diff             0.048660         0.084236            0.578           0.563
L20.forward1yr_diff         0.032958         0.037212            0.886           0.376
L20.ted                     0.023445         0.024090            0.973           0.330
L20.vix                    -0.000927         0.000785           -1.181           0.238
L20.rec_ind                 0.003423         0.030796            0.111           0.911
L20.1yr-ffr                -0.001278         0.004441           -0.288           0.773
L21.yieldsp_diff            0.018034         0.023141            0.779           0.436
L21.termpr_diff             0.105847         0.084227            1.257           0.209
L21.forward1yr_diff        -0.079801         0.037220           -2.144           0.032
L21.ted                    -0.026779         0.024101           -1.111           0.267
L21.vix                     0.000884         0.000785            1.126           0.260
L21.rec_ind                -0.017177         0.030791           -0.558           0.577
L21.1yr-ffr                -0.006078         0.004443           -1.368           0.171
L22.yieldsp_diff           -0.036863         0.023157           -1.592           0.111
L22.termpr_diff             0.166205         0.084296            1.972           0.049
L22.forward1yr_diff        -0.083781         0.037248           -2.249           0.024
L22.ted                     0.019026         0.024110            0.789           0.430
L22.vix                    -0.000433         0.000785           -0.551           0.581
L22.rec_ind                 0.044045         0.030806            1.430           0.153
L22.1yr-ffr                 0.000247         0.004442            0.056           0.956
L23.yieldsp_diff           -0.002755         0.023140           -0.119           0.905
L23.termpr_diff             0.089093         0.084277            1.057           0.290
L23.forward1yr_diff        -0.060898         0.037232           -1.636           0.102
L23.ted                    -0.019246         0.024107           -0.798           0.425
L23.vix                    -0.000328         0.000784           -0.419           0.675
L23.rec_ind                -0.008767         0.030853           -0.284           0.776
L23.1yr-ffr                 0.006518         0.004442            1.467           0.142
L24.yieldsp_diff           -0.034591         0.023133           -1.495           0.135
L24.termpr_diff            -0.063544         0.084218           -0.755           0.451
L24.forward1yr_diff         0.079825         0.037208            2.145           0.032
L24.ted                     0.022458         0.024112            0.931           0.352
L24.vix                     0.000499         0.000783            0.637           0.524
L24.rec_ind                -0.063513         0.030892           -2.056           0.040
L24.1yr-ffr                -0.004992         0.004438           -1.125           0.261
L25.yieldsp_diff           -0.059224         0.023123           -2.561           0.010
L25.termpr_diff             0.175094         0.084107            2.082           0.037
L25.forward1yr_diff        -0.054809         0.037198           -1.473           0.141
L25.ted                     0.014797         0.024110            0.614           0.539
L25.vix                    -0.000584         0.000783           -0.746           0.455
L25.rec_ind                 0.031112         0.030889            1.007           0.314
L25.1yr-ffr                -0.004458         0.004441           -1.004           0.316
L26.yieldsp_diff           -0.028428         0.023132           -1.229           0.219
L26.termpr_diff            -0.047945         0.084162           -0.570           0.569
L26.forward1yr_diff         0.052032         0.037190            1.399           0.162
L26.ted                    -0.011729         0.024115           -0.486           0.627
L26.vix                    -0.000125         0.000782           -0.160           0.873
L26.rec_ind                 0.040289         0.030866            1.305           0.192
L26.1yr-ffr                 0.000284         0.004439            0.064           0.949
L27.yieldsp_diff            0.014202         0.023122            0.614           0.539
L27.termpr_diff            -0.021477         0.084211           -0.255           0.799
L27.forward1yr_diff        -0.017422         0.037168           -0.469           0.639
L27.ted                    -0.000589         0.024095           -0.024           0.980
L27.vix                    -0.000287         0.000782           -0.367           0.713
L27.rec_ind                -0.023865         0.030853           -0.774           0.439
L27.1yr-ffr                 0.000231         0.004438            0.052           0.958
L28.yieldsp_diff            0.074302         0.023116            3.214           0.001
L28.termpr_diff            -0.102781         0.084220           -1.220           0.222
L28.forward1yr_diff        -0.005211         0.037170           -0.140           0.889
L28.ted                    -0.006554         0.024088           -0.272           0.786
L28.vix                    -0.000046         0.000782           -0.058           0.953
L28.rec_ind                 0.036792         0.030836            1.193           0.233
L28.1yr-ffr                 0.001290         0.004435            0.291           0.771
L29.yieldsp_diff           -0.011985         0.023116           -0.518           0.604
L29.termpr_diff            -0.002213         0.084223           -0.026           0.979
L29.forward1yr_diff         0.023207         0.037162            0.624           0.532
L29.ted                     0.017210         0.024073            0.715           0.475
L29.vix                     0.000790         0.000781            1.011           0.312
L29.rec_ind                -0.013742         0.030833           -0.446           0.656
L29.1yr-ffr                -0.000127         0.004457           -0.028           0.977
L30.yieldsp_diff           -0.084394         0.023078           -3.657           0.000
L30.termpr_diff             0.180847         0.084212            2.148           0.032
L30.forward1yr_diff        -0.031353         0.037173           -0.843           0.399
L30.ted                    -0.011324         0.024068           -0.470           0.638
L30.vix                     0.000002         0.000781            0.003           0.998
L30.rec_ind                 0.033456         0.030827            1.085           0.278
L30.1yr-ffr                -0.000650         0.004465           -0.146           0.884
L31.yieldsp_diff           -0.034373         0.023043           -1.492           0.136
L31.termpr_diff             0.039537         0.084217            0.469           0.639
L31.forward1yr_diff        -0.002100         0.037181           -0.056           0.955
L31.ted                    -0.023072         0.024052           -0.959           0.337
L31.vix                    -0.000985         0.000778           -1.266           0.205
L31.rec_ind                -0.127243         0.030831           -4.127           0.000
L31.1yr-ffr                 0.004285         0.004460            0.961           0.337
L32.yieldsp_diff            0.028381         0.023044            1.232           0.218
L32.termpr_diff            -0.133392         0.084139           -1.585           0.113
L32.forward1yr_diff         0.029795         0.037156            0.802           0.423
L32.ted                    -0.021160         0.023963           -0.883           0.377
L32.vix                     0.001250         0.000776            1.611           0.107
L32.rec_ind                 0.039428         0.030868            1.277           0.201
L32.1yr-ffr                -0.001236         0.004421           -0.280           0.780
L33.yieldsp_diff            0.001631         0.023054            0.071           0.944
L33.termpr_diff            -0.040313         0.084175           -0.479           0.632
L33.forward1yr_diff         0.015320         0.037170            0.412           0.680
L33.ted                     0.032155         0.023981            1.341           0.180
L33.vix                    -0.001018         0.000776           -1.312           0.190
L33.rec_ind                 0.056567         0.030863            1.833           0.067
L33.1yr-ffr                -0.000848         0.004418           -0.192           0.848
L34.yieldsp_diff            0.016825         0.023053            0.730           0.465
L34.termpr_diff            -0.040247         0.084126           -0.478           0.632
L34.forward1yr_diff         0.014920         0.037148            0.402           0.688
L34.ted                    -0.010827         0.023996           -0.451           0.652
L34.vix                     0.000805         0.000776            1.038           0.299
L34.rec_ind                -0.041181         0.030852           -1.335           0.182
L34.1yr-ffr                -0.001664         0.004414           -0.377           0.706
L35.yieldsp_diff            0.013990         0.023045            0.607           0.544
L35.termpr_diff             0.010830         0.084160            0.129           0.898
L35.forward1yr_diff        -0.030591         0.037151           -0.823           0.410
L35.ted                    -0.001803         0.024003           -0.075           0.940
L35.vix                    -0.001059         0.000776           -1.365           0.172
L35.rec_ind                 0.005574         0.030836            0.181           0.857
L35.1yr-ffr                 0.009949         0.004416            2.253           0.024
L36.yieldsp_diff           -0.026509         0.023043           -1.150           0.250
L36.termpr_diff            -0.057236         0.084157           -0.680           0.496
L36.forward1yr_diff         0.060177         0.037165            1.619           0.105
L36.ted                     0.047018         0.023981            1.961           0.050
L36.vix                     0.000932         0.000775            1.203           0.229
L36.rec_ind                 0.014418         0.030814            0.468           0.640
L36.1yr-ffr                -0.013047         0.004412           -2.957           0.003
L37.yieldsp_diff           -0.000068         0.023040           -0.003           0.998
L37.termpr_diff            -0.005185         0.084134           -0.062           0.951
L37.forward1yr_diff        -0.018739         0.037155           -0.504           0.614
L37.ted                    -0.032670         0.023964           -1.363           0.173
L37.vix                     0.000301         0.000775            0.389           0.698
L37.rec_ind                -0.002589         0.030815           -0.084           0.933
L37.1yr-ffr                 0.006512         0.004412            1.476           0.140
L38.yieldsp_diff           -0.039120         0.022985           -1.702           0.089
L38.termpr_diff             0.166400         0.083971            1.982           0.048
L38.forward1yr_diff        -0.063927         0.037125           -1.722           0.085
L38.ted                     0.013099         0.023968            0.547           0.585
L38.vix                    -0.001461         0.000775           -1.887           0.059
L38.rec_ind                -0.023665         0.030809           -0.768           0.442
L38.1yr-ffr                 0.000387         0.004413            0.088           0.930
L39.yieldsp_diff           -0.036010         0.022986           -1.567           0.117
L39.termpr_diff             0.007256         0.083906            0.086           0.931
L39.forward1yr_diff         0.024808         0.037102            0.669           0.504
L39.ted                    -0.021340         0.023937           -0.892           0.373
L39.vix                     0.002164         0.000774            2.797           0.005
L39.rec_ind                 0.001690         0.030800            0.055           0.956
L39.1yr-ffr                -0.005366         0.004407           -1.218           0.223
L40.yieldsp_diff            0.007839         0.022925            0.342           0.732
L40.termpr_diff             0.020445         0.083770            0.244           0.807
L40.forward1yr_diff        -0.000556         0.037043           -0.015           0.988
L40.ted                    -0.000209         0.023926           -0.009           0.993
L40.vix                    -0.000732         0.000774           -0.945           0.344
L40.rec_ind                 0.067711         0.030799            2.198           0.028
L40.1yr-ffr                -0.004024         0.004401           -0.914           0.361
L41.yieldsp_diff            0.013775         0.022875            0.602           0.547
L41.termpr_diff            -0.104421         0.083677           -1.248           0.212
L41.forward1yr_diff         0.034127         0.037035            0.921           0.357
L41.ted                    -0.015904         0.023912           -0.665           0.506
L41.vix                    -0.000926         0.000773           -1.197           0.231
L41.rec_ind                -0.050714         0.030806           -1.646           0.100
L41.1yr-ffr                 0.002374         0.004400            0.540           0.590
L42.yieldsp_diff            0.044980         0.022693            1.982           0.047
L42.termpr_diff            -0.008130         0.083130           -0.098           0.922
L42.forward1yr_diff        -0.046082         0.036942           -1.247           0.212
L42.ted                     0.032649         0.023902            1.366           0.172
L42.vix                     0.000933         0.000773            1.208           0.227
L42.rec_ind                 0.009657         0.030802            0.313           0.754
L42.1yr-ffr                -0.001297         0.004396           -0.295           0.768
L43.yieldsp_diff            0.005165         0.022347            0.231           0.817
L43.termpr_diff            -0.056968         0.082200           -0.693           0.488
L43.forward1yr_diff         0.024883         0.036716            0.678           0.498
L43.ted                    -0.016504         0.023887           -0.691           0.490
L43.vix                    -0.000787         0.000766           -1.028           0.304
L43.rec_ind                 0.008788         0.030798            0.285           0.775
L43.1yr-ffr                 0.001866         0.004405            0.424           0.672
L44.yieldsp_diff            0.011626         0.022301            0.521           0.602
L44.termpr_diff            -0.035291         0.082057           -0.430           0.667
L44.forward1yr_diff         0.037846         0.036500            1.037           0.300
L44.ted                     0.003992         0.023852            0.167           0.867
L44.vix                     0.001499         0.000727            2.063           0.039
L44.rec_ind                 0.036863         0.030794            1.197           0.231
L44.1yr-ffr                 0.004604         0.004398            1.047           0.295
L45.yieldsp_diff           -0.033759         0.016804           -2.009           0.045
L45.termpr_diff             0.112549         0.068953            1.632           0.103
L45.forward1yr_diff        -0.011704         0.031514           -0.371           0.710
L45.ted                    -0.002933         0.016850           -0.174           0.862
L45.vix                    -0.001174         0.000454           -2.589           0.010
L45.rec_ind                -0.047616         0.021806           -2.184           0.029
L45.1yr-ffr                -0.006676         0.003740           -1.785           0.074
======================================================================================

Results for equation termpr_diff
======================================================================================
                         coefficient       std. error           t-stat            prob
--------------------------------------------------------------------------------------
const                       0.001174         0.000680            1.728           0.084
L1.yieldsp_diff             0.029469         0.009230            3.193           0.001
L1.termpr_diff              0.215720         0.032307            6.677           0.000
L1.forward1yr_diff         -0.037458         0.013571           -2.760           0.006
L1.ted                      0.000099         0.006901            0.014           0.989
L1.vix                      0.000128         0.000189            0.680           0.497
L1.rec_ind                  0.012063         0.008888            1.357           0.175
L1.1yr-ffr                 -0.000236         0.001549           -0.152           0.879
L2.yieldsp_diff             0.039016         0.009271            4.208           0.000
L2.termpr_diff             -0.128530         0.034176           -3.761           0.000
L2.forward1yr_diff          0.034689         0.015028            2.308           0.021
L2.ted                     -0.012853         0.009742           -1.319           0.187
L2.vix                      0.000150         0.000304            0.494           0.621
L2.rec_ind                 -0.009737         0.012555           -0.776           0.438
L2.1yr-ffr                  0.002497         0.001812            1.378           0.168
L3.yieldsp_diff             0.026067         0.009300            2.803           0.005
L3.termpr_diff              0.028921         0.034315            0.843           0.399
L3.forward1yr_diff         -0.018203         0.015114           -1.204           0.228
L3.ted                      0.004193         0.009739            0.430           0.667
L3.vix                     -0.000126         0.000320           -0.393           0.694
L3.rec_ind                  0.021963         0.012565            1.748           0.080
L3.1yr-ffr                 -0.001858         0.001812           -1.025           0.305
L4.yieldsp_diff             0.011902         0.009411            1.265           0.206
L4.termpr_diff              0.034313         0.034398            0.998           0.319
L4.forward1yr_diff         -0.033042         0.015173           -2.178           0.029
L4.ted                      0.005486         0.009740            0.563           0.573
L4.vix                      0.000211         0.000323            0.653           0.514
L4.rec_ind                 -0.044660         0.012569           -3.553           0.000
L4.1yr-ffr                 -0.000894         0.001809           -0.494           0.621
L5.yieldsp_diff             0.008530         0.009448            0.903           0.367
L5.termpr_diff             -0.018862         0.034439           -0.548           0.584
L5.forward1yr_diff          0.002655         0.015191            0.175           0.861
L5.ted                      0.005743         0.009753            0.589           0.556
L5.vix                     -0.000209         0.000322           -0.648           0.517
L5.rec_ind                  0.003714         0.012577            0.295           0.768
L5.1yr-ffr                  0.001088         0.001809            0.601           0.548
L6.yieldsp_diff             0.005767         0.009485            0.608           0.543
L6.termpr_diff             -0.003839         0.034447           -0.111           0.911
L6.forward1yr_diff         -0.007734         0.015191           -0.509           0.611
L6.ted                     -0.006699         0.009759           -0.686           0.492
L6.vix                      0.000073         0.000322            0.226           0.821
L6.rec_ind                  0.016269         0.012573            1.294           0.196
L6.1yr-ffr                 -0.000776         0.001810           -0.429           0.668
L7.yieldsp_diff             0.021785         0.009490            2.296           0.022
L7.termpr_diff             -0.050482         0.034448           -1.465           0.143
L7.forward1yr_diff          0.004306         0.015194            0.283           0.777
L7.ted                      0.008800         0.009752            0.902           0.367
L7.vix                      0.000268         0.000322            0.833           0.405
L7.rec_ind                  0.012475         0.012575            0.992           0.321
L7.1yr-ffr                 -0.000920         0.001813           -0.507           0.612
L8.yieldsp_diff            -0.001732         0.009504           -0.182           0.855
L8.termpr_diff              0.000574         0.034528            0.017           0.987
L8.forward1yr_diff          0.005213         0.015217            0.343           0.732
L8.ted                     -0.000450         0.009758           -0.046           0.963
L8.vix                     -0.000621         0.000322           -1.929           0.054
L8.rec_ind                 -0.033957         0.012574           -2.701           0.007
L8.1yr-ffr                 -0.002909         0.001817           -1.601           0.109
L9.yieldsp_diff             0.014238         0.009494            1.500           0.134
L9.termpr_diff             -0.046737         0.034551           -1.353           0.176
L9.forward1yr_diff          0.021518         0.015229            1.413           0.158
L9.ted                     -0.015968         0.009769           -1.634           0.102
L9.vix                      0.000415         0.000321            1.292           0.196
L9.rec_ind                  0.044488         0.012580            3.536           0.000
L9.1yr-ffr                  0.004216         0.001816            2.322           0.020
L10.yieldsp_diff            0.001314         0.009498            0.138           0.890
L10.termpr_diff            -0.003407         0.034552           -0.099           0.921
L10.forward1yr_diff        -0.002845         0.015234           -0.187           0.852
L10.ted                     0.009494         0.009776            0.971           0.331
L10.vix                    -0.000385         0.000321           -1.199           0.231
L10.rec_ind                -0.026806         0.012592           -2.129           0.033
L10.1yr-ffr                 0.000874         0.001816            0.481           0.630
L11.yieldsp_diff           -0.003307         0.009490           -0.348           0.728
L11.termpr_diff            -0.000763         0.034537           -0.022           0.982
L11.forward1yr_diff        -0.002585         0.015243           -0.170           0.865
L11.ted                    -0.011530         0.009785           -1.178           0.239
L11.vix                     0.000213         0.000321            0.665           0.506
L11.rec_ind                 0.026001         0.012591            2.065           0.039
L11.1yr-ffr                -0.002222         0.001818           -1.222           0.222
L12.yieldsp_diff            0.014904         0.009492            1.570           0.116
L12.termpr_diff            -0.019155         0.034562           -0.554           0.579
L12.forward1yr_diff        -0.000918         0.015249           -0.060           0.952
L12.ted                     0.000079         0.009790            0.008           0.994
L12.vix                     0.000122         0.000321            0.379           0.705
L12.rec_ind                -0.022436         0.012590           -1.782           0.075
L12.1yr-ffr                 0.001346         0.001817            0.741           0.459
L13.yieldsp_diff           -0.005807         0.009491           -0.612           0.541
L13.termpr_diff             0.035334         0.034562            1.022           0.307
L13.forward1yr_diff        -0.013288         0.015251           -0.871           0.384
L13.ted                     0.018380         0.009793            1.877           0.061
L13.vix                    -0.000448         0.000321           -1.396           0.163
L13.rec_ind                -0.017466         0.012589           -1.387           0.165
L13.1yr-ffr                -0.000800         0.001815           -0.441           0.660
L14.yieldsp_diff           -0.012249         0.009479           -1.292           0.196
L14.termpr_diff             0.087219         0.034528            2.526           0.012
L14.forward1yr_diff        -0.027405         0.015253           -1.797           0.072
L14.ted                     0.004335         0.009800            0.442           0.658
L14.vix                     0.000522         0.000321            1.628           0.103
L14.rec_ind                -0.000652         0.012577           -0.052           0.959
L14.1yr-ffr                 0.002983         0.001815            1.643           0.100
L15.yieldsp_diff            0.008183         0.009483            0.863           0.388
L15.termpr_diff            -0.023096         0.034531           -0.669           0.504
L15.forward1yr_diff        -0.003969         0.015255           -0.260           0.795
L15.ted                    -0.019401         0.009845           -1.971           0.049
L15.vix                    -0.000248         0.000322           -0.772           0.440
L15.rec_ind                 0.031881         0.012564            2.537           0.011
L15.1yr-ffr                -0.002716         0.001831           -1.483           0.138
L16.yieldsp_diff            0.003232         0.009479            0.341           0.733
L16.termpr_diff            -0.016975         0.034534           -0.492           0.623
L16.forward1yr_diff         0.012060         0.015253            0.791           0.429
L16.ted                     0.009628         0.009856            0.977           0.329
L16.vix                     0.000022         0.000322            0.070           0.945
L16.rec_ind                -0.026416         0.012565           -2.102           0.036
L16.1yr-ffr                -0.000415         0.001834           -0.226           0.821
L17.yieldsp_diff            0.006419         0.009478            0.677           0.498
L17.termpr_diff             0.011155         0.034533            0.323           0.747
L17.forward1yr_diff        -0.018799         0.015257           -1.232           0.218
L17.ted                    -0.004160         0.009858           -0.422           0.673
L17.vix                     0.000046         0.000322            0.143           0.887
L17.rec_ind                 0.019065         0.012563            1.518           0.129
L17.1yr-ffr                 0.001890         0.001832            1.032           0.302
L18.yieldsp_diff           -0.011175         0.009486           -1.178           0.239
L18.termpr_diff             0.006024         0.034511            0.175           0.861
L18.forward1yr_diff         0.015639         0.015252            1.025           0.305
L18.ted                     0.007880         0.009857            0.799           0.424
L18.vix                    -0.000150         0.000322           -0.467           0.641
L18.rec_ind                 0.006231         0.012578            0.495           0.620
L18.1yr-ffr                 0.000879         0.001822            0.482           0.630
L19.yieldsp_diff            0.014023         0.009489            1.478           0.139
L19.termpr_diff            -0.057898         0.034502           -1.678           0.093
L19.forward1yr_diff         0.023407         0.015248            1.535           0.125
L19.ted                    -0.003524         0.009868           -0.357           0.721
L19.vix                     0.000272         0.000322            0.845           0.398
L19.rec_ind                -0.015045         0.012591           -1.195           0.232
L19.1yr-ffr                 0.000826         0.001820            0.454           0.650
L20.yieldsp_diff           -0.007621         0.009482           -0.804           0.422
L20.termpr_diff             0.011809         0.034511            0.342           0.732
L20.forward1yr_diff         0.009371         0.015246            0.615           0.539
L20.ted                     0.003270         0.009870            0.331           0.740
L20.vix                    -0.000415         0.000322           -1.292           0.196
L20.rec_ind                 0.006169         0.012617            0.489           0.625
L20.1yr-ffr                -0.001911         0.001820           -1.050           0.294
L21.yieldsp_diff           -0.020284         0.009481           -2.140           0.032
L21.termpr_diff             0.118476         0.034508            3.433           0.001
L21.forward1yr_diff        -0.041664         0.015249           -2.732           0.006
L21.ted                     0.006138         0.009874            0.622           0.534
L21.vix                     0.000370         0.000321            1.149           0.250
L21.rec_ind                -0.014430         0.012615           -1.144           0.253
L21.1yr-ffr                -0.002318         0.001820           -1.273           0.203
L22.yieldsp_diff           -0.016644         0.009488           -1.754           0.079
L22.termpr_diff             0.047163         0.034536            1.366           0.172
L22.forward1yr_diff        -0.021954         0.015260           -1.439           0.150
L22.ted                    -0.007327         0.009878           -0.742           0.458
L22.vix                    -0.000147         0.000321           -0.457           0.647
L22.rec_ind                 0.014589         0.012621            1.156           0.248
L22.1yr-ffr                 0.001412         0.001820            0.776           0.438
L23.yieldsp_diff            0.002812         0.009480            0.297           0.767
L23.termpr_diff             0.006118         0.034528            0.177           0.859
L23.forward1yr_diff         0.001932         0.015254            0.127           0.899
L23.ted                    -0.002218         0.009877           -0.225           0.822
L23.vix                     0.000045         0.000321            0.141           0.888
L23.rec_ind                 0.021844         0.012640            1.728           0.084
L23.1yr-ffr                 0.002352         0.001820            1.292           0.196
L24.yieldsp_diff           -0.000570         0.009477           -0.060           0.952
L24.termpr_diff            -0.037578         0.034504           -1.089           0.276
L24.forward1yr_diff         0.028873         0.015244            1.894           0.058
L24.ted                     0.012773         0.009879            1.293           0.196
L24.vix                    -0.000061         0.000321           -0.190           0.850
L24.rec_ind                -0.045133         0.012656           -3.566           0.000
L24.1yr-ffr                -0.004393         0.001818           -2.416           0.016
L25.yieldsp_diff           -0.001331         0.009473           -0.141           0.888
L25.termpr_diff            -0.002340         0.034458           -0.068           0.946
L25.forward1yr_diff        -0.003612         0.015240           -0.237           0.813
L25.ted                    -0.005968         0.009878           -0.604           0.546
L25.vix                    -0.000168         0.000321           -0.523           0.601
L25.rec_ind                 0.017655         0.012655            1.395           0.163
L25.1yr-ffr                 0.001353         0.001819            0.744           0.457
L26.yieldsp_diff           -0.008955         0.009477           -0.945           0.345
L26.termpr_diff             0.049151         0.034481            1.425           0.154
L26.forward1yr_diff        -0.018475         0.015237           -1.213           0.225
L26.ted                    -0.004178         0.009880           -0.423           0.672
L26.vix                     0.000032         0.000320            0.098           0.922
L26.rec_ind                 0.013943         0.012646            1.103           0.270
L26.1yr-ffr                -0.001843         0.001819           -1.014           0.311
L27.yieldsp_diff            0.010750         0.009473            1.135           0.256
L27.termpr_diff            -0.035105         0.034501           -1.017           0.309
L27.forward1yr_diff         0.009718         0.015228            0.638           0.523
L27.ted                    -0.009994         0.009872           -1.012           0.311
L27.vix                    -0.000146         0.000320           -0.455           0.649
L27.rec_ind                -0.008521         0.012640           -0.674           0.500
L27.1yr-ffr                -0.000434         0.001818           -0.239           0.811
L28.yieldsp_diff            0.012441         0.009471            1.314           0.189
L28.termpr_diff            -0.006453         0.034505           -0.187           0.852
L28.forward1yr_diff        -0.003047         0.015228           -0.200           0.841
L28.ted                     0.009488         0.009869            0.961           0.336
L28.vix                     0.000069         0.000320            0.216           0.829
L28.rec_ind                 0.010045         0.012634            0.795           0.427
L28.1yr-ffr                -0.001371         0.001817           -0.755           0.450
L29.yieldsp_diff            0.004220         0.009471            0.446           0.656
L29.termpr_diff            -0.031975         0.034506           -0.927           0.354
L29.forward1yr_diff         0.019739         0.015225            1.296           0.195
L29.ted                     0.001970         0.009863            0.200           0.842
L29.vix                     0.000183         0.000320            0.572           0.567
L29.rec_ind                -0.001090         0.012632           -0.086           0.931
L29.1yr-ffr                 0.003057         0.001826            1.674           0.094
L30.yieldsp_diff           -0.019142         0.009455           -2.025           0.043
L30.termpr_diff             0.033826         0.034502            0.980           0.327
L30.forward1yr_diff         0.000648         0.015230            0.043           0.966
L30.ted                     0.004198         0.009861            0.426           0.670
L30.vix                     0.000293         0.000320            0.917           0.359
L30.rec_ind                -0.005353         0.012630           -0.424           0.672
L30.1yr-ffr                -0.000331         0.001829           -0.181           0.856
L31.yieldsp_diff            0.003861         0.009441            0.409           0.683
L31.termpr_diff            -0.032791         0.034503           -0.950           0.342
L31.forward1yr_diff         0.010475         0.015233            0.688           0.492
L31.ted                    -0.011547         0.009854           -1.172           0.241
L31.vix                    -0.000794         0.000319           -2.492           0.013
L31.rec_ind                -0.024575         0.012631           -1.946           0.052
L31.1yr-ffr                 0.000705         0.001827            0.386           0.700
L32.yieldsp_diff            0.025346         0.009441            2.685           0.007
L32.termpr_diff            -0.066076         0.034472           -1.917           0.055
L32.forward1yr_diff         0.000780         0.015223            0.051           0.959
L32.ted                     0.000952         0.009818            0.097           0.923
L32.vix                     0.000513         0.000318            1.615           0.106
L32.rec_ind                 0.007990         0.012647            0.632           0.528
L32.1yr-ffr                -0.000227         0.001811           -0.125           0.900
L33.yieldsp_diff            0.004090         0.009445            0.433           0.665
L33.termpr_diff             0.017436         0.034486            0.506           0.613
L33.forward1yr_diff        -0.015924         0.015228           -1.046           0.296
L33.ted                     0.003255         0.009825            0.331           0.740
L33.vix                    -0.000139         0.000318           -0.436           0.663
L33.rec_ind                 0.010361         0.012645            0.819           0.413
L33.1yr-ffr                 0.000645         0.001810            0.356           0.722
L34.yieldsp_diff            0.010617         0.009445            1.124           0.261
L34.termpr_diff            -0.046864         0.034466           -1.360           0.174
L34.forward1yr_diff         0.021000         0.015219            1.380           0.168
L34.ted                    -0.000609         0.009831           -0.062           0.951
L34.vix                    -0.000019         0.000318           -0.059           0.953
L34.rec_ind                -0.005428         0.012640           -0.429           0.668
L34.1yr-ffr                 0.000000         0.001808            0.000           1.000
L35.yieldsp_diff            0.010530         0.009441            1.115           0.265
L35.termpr_diff            -0.022498         0.034480           -0.653           0.514
L35.forward1yr_diff        -0.002414         0.015221           -0.159           0.874
L35.ted                    -0.004301         0.009834           -0.437           0.662
L35.vix                    -0.000212         0.000318           -0.667           0.505
L35.rec_ind                -0.001366         0.012633           -0.108           0.914
L35.1yr-ffr                 0.003201         0.001809            1.769           0.077
L36.yieldsp_diff            0.001076         0.009441            0.114           0.909
L36.termpr_diff            -0.033336         0.034479           -0.967           0.334
L36.forward1yr_diff         0.016510         0.015226            1.084           0.278
L36.ted                     0.009400         0.009825            0.957           0.339
L36.vix                     0.000144         0.000318            0.453           0.650
L36.rec_ind                 0.008933         0.012624            0.708           0.479
L36.1yr-ffr                -0.004723         0.001807           -2.613           0.009
L37.yieldsp_diff            0.000780         0.009439            0.083           0.934
L37.termpr_diff             0.006523         0.034469            0.189           0.850
L37.forward1yr_diff        -0.013768         0.015222           -0.904           0.366
L37.ted                    -0.004338         0.009818           -0.442           0.659
L37.vix                     0.000222         0.000318            0.700           0.484
L37.rec_ind                 0.000241         0.012625            0.019           0.985
L37.1yr-ffr                 0.000086         0.001808            0.048           0.962
L38.yieldsp_diff            0.004612         0.009417            0.490           0.624
L38.termpr_diff             0.014679         0.034402            0.427           0.670
L38.forward1yr_diff        -0.014468         0.015210           -0.951           0.342
L38.ted                     0.000315         0.009820            0.032           0.974
L38.vix                    -0.000691         0.000317           -2.176           0.030
L38.rec_ind                 0.000330         0.012622            0.026           0.979
L38.1yr-ffr                 0.001074         0.001808            0.594           0.552
L39.yieldsp_diff           -0.000177         0.009417           -0.019           0.985
L39.termpr_diff            -0.027055         0.034376           -0.787           0.431
L39.forward1yr_diff         0.019554         0.015201            1.286           0.198
L39.ted                    -0.005176         0.009807           -0.528           0.598
L39.vix                     0.001013         0.000317            3.195           0.001
L39.rec_ind                -0.002688         0.012619           -0.213           0.831
L39.1yr-ffr                -0.000939         0.001805           -0.520           0.603
L40.yieldsp_diff            0.006727         0.009392            0.716           0.474
L40.termpr_diff             0.001366         0.034320            0.040           0.968
L40.forward1yr_diff         0.007884         0.015176            0.520           0.603
L40.ted                    -0.000295         0.009803           -0.030           0.976
L40.vix                    -0.000541         0.000317           -1.708           0.088
L40.rec_ind                 0.026560         0.012618            2.105           0.035
L40.1yr-ffr                -0.002274         0.001803           -1.261           0.207
L41.yieldsp_diff            0.019108         0.009372            2.039           0.041
L41.termpr_diff            -0.077710         0.034282           -2.267           0.023
L41.forward1yr_diff         0.026828         0.015173            1.768           0.077
L41.ted                    -0.017872         0.009797           -1.824           0.068
L41.vix                     0.000178         0.000317            0.562           0.574
L41.rec_ind                -0.020092         0.012621           -1.592           0.111
L41.1yr-ffr                 0.000090         0.001803            0.050           0.960
L42.yieldsp_diff           -0.002161         0.009297           -0.232           0.816
L42.termpr_diff             0.023995         0.034058            0.705           0.481
L42.forward1yr_diff        -0.010466         0.015135           -0.691           0.489
L42.ted                     0.024836         0.009793            2.536           0.011
L42.vix                     0.000076         0.000317            0.239           0.811
L42.rec_ind                -0.008939         0.012620           -0.708           0.479
L42.1yr-ffr                 0.000701         0.001801            0.389           0.697
L43.yieldsp_diff           -0.002303         0.009155           -0.252           0.801
L43.termpr_diff            -0.007527         0.033677           -0.223           0.823
L43.forward1yr_diff         0.001908         0.015042            0.127           0.899
L43.ted                    -0.005212         0.009786           -0.533           0.594
L43.vix                    -0.000327         0.000314           -1.042           0.297
L43.rec_ind                 0.009962         0.012618            0.790           0.430
L43.1yr-ffr                 0.001105         0.001805            0.612           0.540
L44.yieldsp_diff            0.003062         0.009137            0.335           0.738
L44.termpr_diff            -0.021913         0.033618           -0.652           0.515
L44.forward1yr_diff         0.019298         0.014954            1.290           0.197
L44.ted                     0.009766         0.009772            0.999           0.318
L44.vix                     0.000835         0.000298            2.804           0.005
L44.rec_ind                 0.015906         0.012616            1.261           0.207
L44.1yr-ffr                 0.002632         0.001802            1.461           0.144
L45.yieldsp_diff            0.006224         0.006884            0.904           0.366
L45.termpr_diff             0.000329         0.028250            0.012           0.991
L45.forward1yr_diff        -0.003191         0.012911           -0.247           0.805
L45.ted                    -0.008327         0.006903           -1.206           0.228
L45.vix                    -0.000603         0.000186           -3.244           0.001
L45.rec_ind                -0.022360         0.008934           -2.503           0.012
L45.1yr-ffr                -0.002009         0.001532           -1.311           0.190
======================================================================================

Results for equation forward1yr_diff
======================================================================================
                         coefficient       std. error           t-stat            prob
--------------------------------------------------------------------------------------
const                       0.001429         0.001163            1.228           0.219
L1.yieldsp_diff             0.017281         0.015796            1.094           0.274
L1.termpr_diff             -0.341492         0.055288           -6.177           0.000
L1.forward1yr_diff          0.374861         0.023224           16.141           0.000
L1.ted                      0.015688         0.011810            1.328           0.184
L1.vix                     -0.000211         0.000324           -0.652           0.515
L1.rec_ind                  0.027034         0.015210            1.777           0.076
L1.1yr-ffr                 -0.001333         0.002650           -0.503           0.615
L2.yieldsp_diff             0.061983         0.015866            3.907           0.000
L2.termpr_diff             -0.083487         0.058487           -1.427           0.153
L2.forward1yr_diff         -0.024445         0.025717           -0.951           0.342
L2.ted                     -0.013882         0.016671           -0.833           0.405
L2.vix                      0.000966         0.000520            1.859           0.063
L2.rec_ind                 -0.054849         0.021486           -2.553           0.011
L2.1yr-ffr                  0.005406         0.003101            1.743           0.081
L3.yieldsp_diff             0.030895         0.015915            1.941           0.052
L3.termpr_diff             -0.086014         0.058724           -1.465           0.143
L3.forward1yr_diff          0.064680         0.025865            2.501           0.012
L3.ted                      0.001932         0.016667            0.116           0.908
L3.vix                     -0.000807         0.000547           -1.474           0.140
L3.rec_ind                  0.049036         0.021504            2.280           0.023
L3.1yr-ffr                 -0.004299         0.003101           -1.386           0.166
L4.yieldsp_diff             0.039130         0.016106            2.430           0.015
L4.termpr_diff             -0.099047         0.058868           -1.683           0.092
L4.forward1yr_diff          0.001543         0.025966            0.059           0.953
L4.ted                     -0.018015         0.016668           -1.081           0.280
L4.vix                      0.000309         0.000552            0.559           0.576
L4.rec_ind                 -0.043682         0.021510           -2.031           0.042
L4.1yr-ffr                  0.000007         0.003096            0.002           0.998
L5.yieldsp_diff             0.047063         0.016168            2.911           0.004
L5.termpr_diff             -0.109885         0.058937           -1.864           0.062
L5.forward1yr_diff          0.025553         0.025997            0.983           0.326
L5.ted                      0.020540         0.016691            1.231           0.218
L5.vix                     -0.000220         0.000552           -0.399           0.690
L5.rec_ind                 -0.012403         0.021523           -0.576           0.564
L5.1yr-ffr                 -0.001890         0.003095           -0.611           0.542
L6.yieldsp_diff             0.042834         0.016231            2.639           0.008
L6.termpr_diff             -0.085900         0.058950           -1.457           0.145
L6.forward1yr_diff          0.006041         0.025997            0.232           0.816
L6.ted                     -0.014272         0.016701           -0.855           0.393
L6.vix                     -0.000158         0.000551           -0.286           0.775
L6.rec_ind                  0.047471         0.021517            2.206           0.027
L6.1yr-ffr                 -0.001629         0.003097           -0.526           0.599
L7.yieldsp_diff             0.073461         0.016241            4.523           0.000
L7.termpr_diff             -0.218578         0.058952           -3.708           0.000
L7.forward1yr_diff          0.058574         0.026002            2.253           0.024
L7.ted                      0.008986         0.016688            0.538           0.590
L7.vix                      0.000518         0.000551            0.941           0.347
L7.rec_ind                  0.010064         0.021520            0.468           0.640
L7.1yr-ffr                 -0.000272         0.003103           -0.088           0.930
L8.yieldsp_diff             0.000198         0.016265            0.012           0.990
L8.termpr_diff              0.070928         0.059089            1.200           0.230
L8.forward1yr_diff         -0.011589         0.026042           -0.445           0.656
L8.ted                      0.015910         0.016700            0.953           0.341
L8.vix                     -0.000551         0.000550           -1.001           0.317
L8.rec_ind                 -0.048745         0.021518           -2.265           0.023
L8.1yr-ffr                 -0.003002         0.003109           -0.966           0.334
L9.yieldsp_diff             0.028233         0.016248            1.738           0.082
L9.termpr_diff             -0.026154         0.059128           -0.442           0.658
L9.forward1yr_diff          0.002019         0.026063            0.077           0.938
L9.ted                     -0.050649         0.016719           -3.029           0.002
L9.vix                      0.000582         0.000550            1.059           0.290
L9.rec_ind                  0.052538         0.021529            2.440           0.015
L9.1yr-ffr                  0.007062         0.003107            2.273           0.023
L10.yieldsp_diff           -0.004650         0.016254           -0.286           0.775
L10.termpr_diff             0.002529         0.059130            0.043           0.966
L10.forward1yr_diff         0.009172         0.026071            0.352           0.725
L10.ted                     0.033681         0.016731            2.013           0.044
L10.vix                    -0.000838         0.000550           -1.523           0.128
L10.rec_ind                -0.019557         0.021549           -0.908           0.364
L10.1yr-ffr                 0.000673         0.003109            0.216           0.829
L11.yieldsp_diff            0.005643         0.016240            0.347           0.728
L11.termpr_diff            -0.035386         0.059105           -0.599           0.549
L11.forward1yr_diff         0.005899         0.026086            0.226           0.821
L11.ted                    -0.011172         0.016745           -0.667           0.505
L11.vix                     0.000574         0.000550            1.044           0.296
L11.rec_ind                 0.024088         0.021547            1.118           0.264
L11.1yr-ffr                -0.001615         0.003111           -0.519           0.604
L12.yieldsp_diff            0.034618         0.016245            2.131           0.033
L12.termpr_diff            -0.060032         0.059148           -1.015           0.310
L12.forward1yr_diff         0.009917         0.026096            0.380           0.704
L12.ted                     0.004193         0.016754            0.250           0.802
L12.vix                     0.000193         0.000549            0.351           0.726
L12.rec_ind                -0.033231         0.021546           -1.542           0.123
L12.1yr-ffr                -0.000322         0.003109           -0.104           0.917
L13.yieldsp_diff            0.007274         0.016242            0.448           0.654
L13.termpr_diff             0.018009         0.059147            0.304           0.761
L13.forward1yr_diff        -0.007832         0.026099           -0.300           0.764
L13.ted                     0.023809         0.016760            1.421           0.155
L13.vix                    -0.000995         0.000549           -1.812           0.070
L13.rec_ind                -0.024324         0.021545           -1.129           0.259
L13.1yr-ffr                 0.001143         0.003106            0.368           0.713
L14.yieldsp_diff           -0.027075         0.016222           -1.669           0.095
L14.termpr_diff             0.108387         0.059090            1.834           0.067
L14.forward1yr_diff        -0.007483         0.026103           -0.287           0.774
L14.ted                     0.001072         0.016772            0.064           0.949
L14.vix                     0.001024         0.000549            1.866           0.062
L14.rec_ind                 0.000502         0.021524            0.023           0.981
L14.1yr-ffr                 0.002494         0.003106            0.803           0.422
L15.yieldsp_diff            0.006479         0.016229            0.399           0.690
L15.termpr_diff            -0.020125         0.059095           -0.341           0.733
L15.forward1yr_diff        -0.011728         0.026107           -0.449           0.653
L15.ted                    -0.039344         0.016849           -2.335           0.020
L15.vix                    -0.000291         0.000550           -0.529           0.597
L15.rec_ind                 0.045087         0.021502            2.097           0.036
L15.1yr-ffr                -0.005220         0.003133           -1.666           0.096
L16.yieldsp_diff            0.022737         0.016223            1.402           0.161
L16.termpr_diff            -0.039977         0.059100           -0.676           0.499
L16.forward1yr_diff         0.015694         0.026103            0.601           0.548
L16.ted                     0.000900         0.016867            0.053           0.957
L16.vix                    -0.000112         0.000552           -0.202           0.840
L16.rec_ind                -0.031454         0.021503           -1.463           0.144
L16.1yr-ffr                -0.002111         0.003138           -0.673           0.501
L17.yieldsp_diff            0.021737         0.016221            1.340           0.180
L17.termpr_diff             0.041850         0.059098            0.708           0.479
L17.forward1yr_diff        -0.040426         0.026110           -1.548           0.122
L17.ted                     0.002022         0.016871            0.120           0.905
L17.vix                    -0.000025         0.000551           -0.045           0.964
L17.rec_ind                -0.017175         0.021500           -0.799           0.424
L17.1yr-ffr                 0.004439         0.003135            1.416           0.157
L18.yieldsp_diff           -0.021912         0.016233           -1.350           0.177
L18.termpr_diff             0.021531         0.059061            0.365           0.715
L18.forward1yr_diff         0.025519         0.026101            0.978           0.328
L18.ted                     0.021772         0.016868            1.291           0.197
L18.vix                     0.000151         0.000551            0.274           0.784
L18.rec_ind                 0.000347         0.021526            0.016           0.987
L18.1yr-ffr                 0.003504         0.003117            1.124           0.261
L19.yieldsp_diff            0.014942         0.016238            0.920           0.357
L19.termpr_diff            -0.038878         0.059046           -0.658           0.510
L19.forward1yr_diff         0.012636         0.026095            0.484           0.628
L19.ted                     0.000216         0.016887            0.013           0.990
L19.vix                     0.000081         0.000550            0.147           0.883
L19.rec_ind                 0.029395         0.021547            1.364           0.172
L19.1yr-ffr                -0.000007         0.003115           -0.002           0.998
L20.yieldsp_diff           -0.016780         0.016227           -1.034           0.301
L20.termpr_diff             0.013843         0.059060            0.234           0.815
L20.forward1yr_diff         0.026659         0.026091            1.022           0.307
L20.ted                    -0.001033         0.016890           -0.061           0.951
L20.vix                    -0.000591         0.000550           -1.074           0.283
L20.rec_ind                -0.005804         0.021592           -0.269           0.788
L20.1yr-ffr                -0.003898         0.003114           -1.252           0.211
L21.yieldsp_diff           -0.038343         0.016225           -2.363           0.018
L21.termpr_diff             0.141957         0.059054            2.404           0.016
L21.forward1yr_diff        -0.032028         0.026096           -1.227           0.220
L21.ted                     0.011567         0.016898            0.685           0.494
L21.vix                     0.000564         0.000550            1.026           0.305
L21.rec_ind                 0.002596         0.021589            0.120           0.904
L21.1yr-ffr                -0.005038         0.003115           -1.617           0.106
L22.yieldsp_diff           -0.035997         0.016236           -2.217           0.027
L22.termpr_diff             0.091694         0.059102            1.551           0.121
L22.forward1yr_diff        -0.026920         0.026116           -1.031           0.303
L22.ted                    -0.004555         0.016904           -0.269           0.788
L22.vix                    -0.000296         0.000550           -0.537           0.591
L22.rec_ind                -0.012384         0.021599           -0.573           0.566
L22.1yr-ffr                 0.007420         0.003114            2.383           0.017
L23.yieldsp_diff           -0.008272         0.016224           -0.510           0.610
L23.termpr_diff             0.012110         0.059090            0.205           0.838
L23.forward1yr_diff         0.018986         0.026104            0.727           0.467
L23.ted                    -0.008151         0.016902           -0.482           0.630
L23.vix                     0.000172         0.000550            0.313           0.754
L23.rec_ind                 0.064567         0.021632            2.985           0.003
L23.1yr-ffr                -0.000532         0.003115           -0.171           0.864
L24.yieldsp_diff           -0.003677         0.016219           -0.227           0.821
L24.termpr_diff            -0.040871         0.059048           -0.692           0.489
L24.forward1yr_diff         0.039289         0.026087            1.506           0.132
L24.ted                     0.028559         0.016906            1.689           0.091
L24.vix                    -0.000006         0.000549           -0.010           0.992
L24.rec_ind                -0.069515         0.021659           -3.210           0.001
L24.1yr-ffr                -0.006612         0.003111           -2.125           0.034
L25.yieldsp_diff            0.017093         0.016212            1.054           0.292
L25.termpr_diff            -0.076510         0.058970           -1.297           0.194
L25.forward1yr_diff         0.018067         0.026081            0.693           0.488
L25.ted                    -0.027363         0.016904           -1.619           0.106
L25.vix                    -0.000418         0.000549           -0.762           0.446
L25.rec_ind                 0.017329         0.021657            0.800           0.424
L25.1yr-ffr                 0.005602         0.003114            1.799           0.072
L26.yieldsp_diff           -0.024750         0.016219           -1.526           0.127
L26.termpr_diff             0.167667         0.059008            2.841           0.004
L26.forward1yr_diff        -0.070403         0.026075           -2.700           0.007
L26.ted                     0.010155         0.016907            0.601           0.548
L26.vix                     0.000302         0.000548            0.550           0.582
L26.rec_ind                 0.030704         0.021641            1.419           0.156
L26.1yr-ffr                -0.005342         0.003112           -1.716           0.086
L27.yieldsp_diff            0.012825         0.016212            0.791           0.429
L27.termpr_diff            -0.096752         0.059043           -1.639           0.101
L27.forward1yr_diff         0.044863         0.026060            1.722           0.085
L27.ted                    -0.031183         0.016894           -1.846           0.065
L27.vix                    -0.000308         0.000548           -0.562           0.574
L27.rec_ind                -0.016391         0.021632           -0.758           0.449
L27.1yr-ffr                -0.001343         0.003112           -0.432           0.666
L28.yieldsp_diff            0.030127         0.016208            1.859           0.063
L28.termpr_diff            -0.064180         0.059049           -1.087           0.277
L28.forward1yr_diff         0.020078         0.026061            0.770           0.441
L28.ted                     0.002515         0.016889            0.149           0.882
L28.vix                     0.000221         0.000548            0.404           0.686
L28.rec_ind                 0.004413         0.021620            0.204           0.838
L28.1yr-ffr                -0.004140         0.003110           -1.331           0.183
L29.yieldsp_diff            0.014604         0.016208            0.901           0.368
L29.termpr_diff            -0.077980         0.059051           -1.321           0.187
L29.forward1yr_diff         0.049540         0.026055            1.901           0.057
L29.ted                     0.009984         0.016878            0.592           0.554
L29.vix                     0.000409         0.000548            0.747           0.455
L29.rec_ind                -0.002885         0.021618           -0.133           0.894
L29.1yr-ffr                 0.006296         0.003125            2.015           0.044
L30.yieldsp_diff           -0.015156         0.016181           -0.937           0.349
L30.termpr_diff             0.045056         0.059044            0.763           0.445
L30.forward1yr_diff         0.002029         0.026063            0.078           0.938
L30.ted                     0.021137         0.016875            1.253           0.210
L30.vix                    -0.000117         0.000547           -0.213           0.831
L30.rec_ind                -0.025755         0.021613           -1.192           0.233
L30.1yr-ffr                 0.002075         0.003130            0.663           0.508
L31.yieldsp_diff            0.025475         0.016156            1.577           0.115
L31.termpr_diff            -0.117884         0.059047           -1.996           0.046
L31.forward1yr_diff         0.045683         0.026069            1.752           0.080
L31.ted                    -0.020763         0.016864           -1.231           0.218
L31.vix                    -0.000482         0.000545           -0.883           0.377
L31.rec_ind                -0.015027         0.021616           -0.695           0.487
L31.1yr-ffr                -0.001495         0.003127           -0.478           0.633
L32.yieldsp_diff            0.035578         0.016157            2.202           0.028
L32.termpr_diff            -0.060414         0.058993           -1.024           0.306
L32.forward1yr_diff        -0.014151         0.026051           -0.543           0.587
L32.ted                     0.003481         0.016802            0.207           0.836
L32.vix                     0.000142         0.000544            0.260           0.795
L32.rec_ind                 0.020371         0.021643            0.941           0.347
L32.1yr-ffr                 0.003019         0.003099            0.974           0.330
L33.yieldsp_diff           -0.006924         0.016164           -0.428           0.668
L33.termpr_diff             0.108599         0.059018            1.840           0.066
L33.forward1yr_diff        -0.061488         0.026061           -2.359           0.018
L33.ted                     0.005187         0.016814            0.309           0.758
L33.vix                     0.000359         0.000544            0.660           0.509
L33.rec_ind                 0.002475         0.021639            0.114           0.909
L33.1yr-ffr                 0.000292         0.003098            0.094           0.925
L34.yieldsp_diff            0.008273         0.016163            0.512           0.609
L34.termpr_diff            -0.097959         0.058983           -1.661           0.097
L34.forward1yr_diff         0.059556         0.026045            2.287           0.022
L34.ted                     0.002825         0.016825            0.168           0.867
L34.vix                    -0.000164         0.000544           -0.302           0.763
L34.rec_ind                -0.004640         0.021632           -0.215           0.830
L34.1yr-ffr                -0.000405         0.003095           -0.131           0.896
L35.yieldsp_diff            0.004969         0.016157            0.308           0.758
L35.termpr_diff             0.005369         0.059007            0.091           0.928
L35.forward1yr_diff        -0.021369         0.026048           -0.820           0.412
L35.ted                    -0.004328         0.016829           -0.257           0.797
L35.vix                    -0.000096         0.000544           -0.176           0.860
L35.rec_ind                -0.012384         0.021620           -0.573           0.567
L35.1yr-ffr                 0.006165         0.003096            1.991           0.046
L36.yieldsp_diff           -0.000826         0.016156           -0.051           0.959
L36.termpr_diff            -0.020245         0.059005           -0.343           0.732
L36.forward1yr_diff         0.009287         0.026058            0.356           0.722
L36.ted                     0.003442         0.016814            0.205           0.838
L36.vix                    -0.000182         0.000543           -0.335           0.738
L36.rec_ind                 0.025805         0.021604            1.194           0.232
L36.1yr-ffr                -0.006121         0.003093           -1.979           0.048
L37.yieldsp_diff            0.004893         0.016154            0.303           0.762
L37.termpr_diff            -0.018159         0.058989           -0.308           0.758
L37.forward1yr_diff        -0.006847         0.026051           -0.263           0.793
L37.ted                    -0.003986         0.016802           -0.237           0.812
L37.vix                     0.000314         0.000543            0.577           0.564
L37.rec_ind                -0.010076         0.021605           -0.466           0.641
L37.1yr-ffr                -0.002106         0.003093           -0.681           0.496
L38.yieldsp_diff            0.012645         0.016115            0.785           0.433
L38.termpr_diff             0.026826         0.058874            0.456           0.649
L38.forward1yr_diff        -0.026271         0.026030           -1.009           0.313
L38.ted                    -0.006598         0.016805           -0.393           0.695
L38.vix                    -0.000741         0.000543           -1.364           0.173
L38.rec_ind                 0.012040         0.021601            0.557           0.577
L38.1yr-ffr                 0.002601         0.003094            0.840           0.401
L39.yieldsp_diff            0.014018         0.016116            0.870           0.384
L39.termpr_diff            -0.080237         0.058829           -1.364           0.173
L39.forward1yr_diff         0.044814         0.026013            1.723           0.085
L39.ted                    -0.002250         0.016783           -0.134           0.893
L39.vix                     0.000977         0.000543            1.801           0.072
L39.rec_ind                 0.005206         0.021595            0.241           0.809
L39.1yr-ffr                -0.002492         0.003090           -0.806           0.420
L40.yieldsp_diff            0.010823         0.016073            0.673           0.501
L40.termpr_diff             0.012950         0.058734            0.220           0.825
L40.forward1yr_diff        -0.000011         0.025972           -0.000           1.000
L40.ted                     0.001606         0.016776            0.096           0.924
L40.vix                    -0.000645         0.000543           -1.189           0.235
L40.rec_ind                 0.038716         0.021594            1.793           0.073
L40.1yr-ffr                -0.004407         0.003086           -1.428           0.153
L41.yieldsp_diff            0.038168         0.016039            2.380           0.017
L41.termpr_diff            -0.138494         0.058669           -2.361           0.018
L41.forward1yr_diff         0.057725         0.025966            2.223           0.026
L41.ted                    -0.039993         0.016765           -2.385           0.017
L41.vix                     0.000707         0.000542            1.304           0.192
L41.rec_ind                -0.028518         0.021599           -1.320           0.187
L41.1yr-ffr                -0.000423         0.003085           -0.137           0.891
L42.yieldsp_diff            0.005004         0.015911            0.315           0.753
L42.termpr_diff             0.000866         0.058285            0.015           0.988
L42.forward1yr_diff         0.001391         0.025901            0.054           0.957
L42.ted                     0.035662         0.016758            2.128           0.033
L42.vix                    -0.000215         0.000542           -0.396           0.692
L42.rec_ind                -0.028005         0.021597           -1.297           0.195
L42.1yr-ffr                 0.001456         0.003082            0.473           0.637
L43.yieldsp_diff           -0.003613         0.015668           -0.231           0.818
L43.termpr_diff             0.033491         0.057633            0.581           0.561
L43.forward1yr_diff        -0.020601         0.025743           -0.800           0.424
L43.ted                     0.001972         0.016748            0.118           0.906
L43.vix                    -0.000447         0.000537           -0.833           0.405
L43.rec_ind                 0.007798         0.021593            0.361           0.718
L43.1yr-ffr                 0.003118         0.003089            1.009           0.313
L44.yieldsp_diff            0.001164         0.015636            0.074           0.941
L44.termpr_diff            -0.003305         0.057533           -0.057           0.954
L44.forward1yr_diff         0.011892         0.025592            0.465           0.642
L44.ted                     0.022193         0.016724            1.327           0.185
L44.vix                     0.001275         0.000510            2.502           0.012
L44.rec_ind                 0.022722         0.021591            1.052           0.293
L44.1yr-ffr                 0.004736         0.003083            1.536           0.125
L45.yieldsp_diff            0.010393         0.011782            0.882           0.378
L45.termpr_diff            -0.041264         0.048345           -0.854           0.393
L45.forward1yr_diff         0.016183         0.022096            0.732           0.464
L45.ted                    -0.015092         0.011814           -1.277           0.201
L45.vix                    -0.000980         0.000318           -3.081           0.002
L45.rec_ind                -0.023381         0.015289           -1.529           0.126
L45.1yr-ffr                -0.001799         0.002622           -0.686           0.493
======================================================================================

Results for equation ted
======================================================================================
                         coefficient       std. error           t-stat            prob
--------------------------------------------------------------------------------------
const                       0.004166         0.001331            3.130           0.002
L1.yieldsp_diff             0.128381         0.018080            7.101           0.000
L1.termpr_diff             -0.386650         0.063281           -6.110           0.000
L1.forward1yr_diff          0.293410         0.026582           11.038           0.000
L1.ted                      1.007447         0.013517           74.532           0.000
L1.vix                      0.002018         0.000370            5.448           0.000
L1.rec_ind                 -0.004816         0.017409           -0.277           0.782
L1.1yr-ffr                 -0.004553         0.003033           -1.501           0.133
L2.yieldsp_diff            -0.031089         0.018159           -1.712           0.087
L2.termpr_diff              0.245509         0.066943            3.667           0.000
L2.forward1yr_diff         -0.129879         0.029435           -4.412           0.000
L2.ted                     -0.055010         0.019082           -2.883           0.004
L2.vix                     -0.002389         0.000595           -4.018           0.000
L2.rec_ind                  0.052501         0.024592            2.135           0.033
L2.1yr-ffr                  0.003753         0.003550            1.057           0.290
L3.yieldsp_diff             0.005383         0.018216            0.295           0.768
L3.termpr_diff             -0.093460         0.067215           -1.390           0.164
L3.forward1yr_diff          0.059362         0.029605            2.005           0.045
L3.ted                      0.005198         0.019076            0.272           0.785
L3.vix                      0.001572         0.000627            2.510           0.012
L3.rec_ind                 -0.069429         0.024613           -2.821           0.005
L3.1yr-ffr                 -0.000952         0.003550           -0.268           0.789
L4.yieldsp_diff            -0.051722         0.018434           -2.806           0.005
L4.termpr_diff              0.248074         0.067378            3.682           0.000
L4.forward1yr_diff         -0.064828         0.029720           -2.181           0.029
L4.ted                      0.037522         0.019078            1.967           0.049
L4.vix                     -0.000027         0.000632           -0.043           0.966
L4.rec_ind                  0.012075         0.024620            0.490           0.624
L4.1yr-ffr                 -0.000380         0.003543           -0.107           0.915
L5.yieldsp_diff            -0.134557         0.018506           -7.271           0.000
L5.termpr_diff              0.195280         0.067458            2.895           0.004
L5.forward1yr_diff          0.035089         0.029755            1.179           0.238
L5.ted                      0.037163         0.019104            1.945           0.052
L5.vix                     -0.000827         0.000631           -1.309           0.190
L5.rec_ind                  0.048734         0.024635            1.978           0.048
L5.1yr-ffr                  0.007515         0.003543            2.121           0.034
L6.yieldsp_diff            -0.089614         0.018578           -4.824           0.000
L6.termpr_diff              0.113034         0.067473            1.675           0.094
L6.forward1yr_diff         -0.010765         0.029755           -0.362           0.717
L6.ted                      0.012521         0.019116            0.655           0.512
L6.vix                      0.000215         0.000631            0.341           0.733
L6.rec_ind                 -0.061262         0.024628           -2.487           0.013
L6.1yr-ffr                 -0.014488         0.003545           -4.087           0.000
L7.yieldsp_diff            -0.093672         0.018589           -5.039           0.000
L7.termpr_diff              0.253844         0.067475            3.762           0.000
L7.forward1yr_diff         -0.061759         0.029761           -2.075           0.038
L7.ted                      0.022911         0.019101            1.199           0.230
L7.vix                     -0.000509         0.000630           -0.808           0.419
L7.rec_ind                  0.045011         0.024631            1.827           0.068
L7.1yr-ffr                  0.010322         0.003551            2.907           0.004
L8.yieldsp_diff            -0.029643         0.018616           -1.592           0.111
L8.termpr_diff             -0.010412         0.067632           -0.154           0.878
L8.forward1yr_diff         -0.042795         0.029807           -1.436           0.151
L8.ted                     -0.100557         0.019114           -5.261           0.000
L8.vix                     -0.000003         0.000630           -0.004           0.997
L8.rec_ind                 -0.036462         0.024629           -1.480           0.139
L8.1yr-ffr                  0.000501         0.003559            0.141           0.888
L9.yieldsp_diff            -0.075320         0.018597           -4.050           0.000
L9.termpr_diff              0.012428         0.067677            0.184           0.854
L9.forward1yr_diff          0.078061         0.029830            2.617           0.009
L9.ted                      0.076881         0.019136            4.018           0.000
L9.vix                      0.000565         0.000630            0.898           0.369
L9.rec_ind                  0.014321         0.024642            0.581           0.561
L9.1yr-ffr                  0.000336         0.003557            0.094           0.925
L10.yieldsp_diff            0.023841         0.018604            1.282           0.200
L10.termpr_diff            -0.026072         0.067678           -0.385           0.700
L10.forward1yr_diff        -0.044366         0.029841           -1.487           0.137
L10.ted                    -0.065371         0.019150           -3.414           0.001
L10.vix                    -0.000386         0.000629           -0.613           0.540
L10.rec_ind                -0.020257         0.024665           -0.821           0.411
L10.1yr-ffr                -0.000521         0.003558           -0.146           0.884
L11.yieldsp_diff            0.014067         0.018588            0.757           0.449
L11.termpr_diff            -0.100085         0.067650           -1.479           0.139
L11.forward1yr_diff         0.037956         0.029857            1.271           0.204
L11.ted                     0.002737         0.019166            0.143           0.886
L11.vix                    -0.000391         0.000629           -0.621           0.534
L11.rec_ind                -0.002776         0.024662           -0.113           0.910
L11.1yr-ffr                -0.007518         0.003561           -2.111           0.035
L12.yieldsp_diff            0.021469         0.018593            1.155           0.248
L12.termpr_diff            -0.071290         0.067699           -1.053           0.292
L12.forward1yr_diff         0.025268         0.029868            0.846           0.398
L12.ted                    -0.024957         0.019176           -1.302           0.193
L12.vix                    -0.000532         0.000629           -0.846           0.398
L12.rec_ind                 0.036097         0.024661            1.464           0.143
L12.1yr-ffr                 0.003252         0.003558            0.914           0.361
L13.yieldsp_diff           -0.004377         0.018590           -0.235           0.814
L13.termpr_diff             0.110793         0.067698            1.637           0.102
L13.forward1yr_diff        -0.082276         0.029872           -2.754           0.006
L13.ted                    -0.018322         0.019183           -0.955           0.340
L13.vix                     0.001142         0.000628            1.818           0.069
L13.rec_ind                -0.018426         0.024660           -0.747           0.455
L13.1yr-ffr                 0.002799         0.003555            0.787           0.431
L14.yieldsp_diff           -0.029066         0.018568           -1.565           0.117
L14.termpr_diff             0.028959         0.067633            0.428           0.669
L14.forward1yr_diff         0.034914         0.029877            1.169           0.243
L14.ted                     0.114168         0.019196            5.947           0.000
L14.vix                    -0.000526         0.000628           -0.838           0.402
L14.rec_ind                -0.001115         0.024636           -0.045           0.964
L14.1yr-ffr                 0.005318         0.003555            1.496           0.135
L15.yieldsp_diff            0.031803         0.018576            1.712           0.087
L15.termpr_diff            -0.110899         0.067639           -1.640           0.101
L15.forward1yr_diff        -0.014271         0.029881           -0.478           0.633
L15.ted                    -0.059471         0.019285           -3.084           0.002
L15.vix                     0.000045         0.000630            0.072           0.943
L15.rec_ind                 0.001937         0.024611            0.079           0.937
L15.1yr-ffr                 0.000326         0.003586            0.091           0.928
L16.yieldsp_diff           -0.035572         0.018568           -1.916           0.055
L16.termpr_diff             0.079076         0.067644            1.169           0.242
L16.forward1yr_diff        -0.002099         0.029877           -0.070           0.944
L16.ted                     0.034724         0.019305            1.799           0.072
L16.vix                     0.000481         0.000631            0.763           0.446
L16.rec_ind                 0.014628         0.024611            0.594           0.552
L16.1yr-ffr                 0.002733         0.003592            0.761           0.447
L17.yieldsp_diff            0.018408         0.018566            0.992           0.321
L17.termpr_diff            -0.072401         0.067642           -1.070           0.284
L17.forward1yr_diff        -0.017012         0.029884           -0.569           0.569
L17.ted                    -0.061295         0.019310           -3.174           0.002
L17.vix                    -0.000011         0.000631           -0.017           0.987
L17.rec_ind                 0.068875         0.024609            2.799           0.005
L17.1yr-ffr                -0.008744         0.003588           -2.437           0.015
L18.yieldsp_diff            0.020384         0.018580            1.097           0.273
L18.termpr_diff            -0.053515         0.067599           -0.792           0.429
L18.forward1yr_diff         0.014461         0.029875            0.484           0.628
L18.ted                     0.021799         0.019307            1.129           0.259
L18.vix                    -0.000202         0.000630           -0.321           0.748
L18.rec_ind                -0.075716         0.024638           -3.073           0.002
L18.1yr-ffr                -0.002211         0.003568           -0.620           0.535
L19.yieldsp_diff            0.009704         0.018586            0.522           0.602
L19.termpr_diff            -0.103047         0.067582           -1.525           0.127
L19.forward1yr_diff         0.042749         0.029868            1.431           0.152
L19.ted                    -0.006254         0.019329           -0.324           0.746
L19.vix                     0.000612         0.000630            0.972           0.331
L19.rec_ind                 0.011822         0.024662            0.479           0.632
L19.1yr-ffr                -0.000330         0.003565           -0.093           0.926
L20.yieldsp_diff            0.012215         0.018573            0.658           0.511
L20.termpr_diff            -0.053041         0.067599           -0.785           0.433
L20.forward1yr_diff         0.032889         0.029863            1.101           0.271
L20.ted                     0.018667         0.019332            0.966           0.334
L20.vix                    -0.001004         0.000630           -1.593           0.111
L20.rec_ind                -0.017066         0.024714           -0.691           0.490
L20.1yr-ffr                 0.007561         0.003564            2.121           0.034
L21.yieldsp_diff            0.034485         0.018570            1.857           0.063
L21.termpr_diff            -0.040301         0.067592           -0.596           0.551
L21.forward1yr_diff         0.004608         0.029869            0.154           0.877
L21.ted                     0.004482         0.019341            0.232           0.817
L21.vix                     0.000943         0.000630            1.497           0.134
L21.rec_ind                 0.002151         0.024710            0.087           0.931
L21.1yr-ffr                -0.005818         0.003565           -1.632           0.103
L22.yieldsp_diff           -0.015382         0.018584           -0.828           0.408
L22.termpr_diff             0.068918         0.067647            1.019           0.308
L22.forward1yr_diff        -0.010274         0.029892           -0.344           0.731
L22.ted                     0.030570         0.019348            1.580           0.114
L22.vix                    -0.000620         0.000630           -0.985           0.325
L22.rec_ind                -0.029333         0.024721           -1.187           0.235
L22.1yr-ffr                -0.006542         0.003565           -1.835           0.066
L23.yieldsp_diff            0.035413         0.018569            1.907           0.057
L23.termpr_diff            -0.008410         0.067632           -0.124           0.901
L23.forward1yr_diff        -0.068436         0.029878           -2.290           0.022
L23.ted                    -0.061074         0.019346           -3.157           0.002
L23.vix                     0.000459         0.000629            0.729           0.466
L23.rec_ind                -0.002866         0.024760           -0.116           0.908
L23.1yr-ffr                 0.008607         0.003565            2.414           0.016
L24.yieldsp_diff           -0.024775         0.018564           -1.335           0.182
L24.termpr_diff            -0.021204         0.067585           -0.314           0.754
L24.forward1yr_diff         0.042048         0.029859            1.408           0.159
L24.ted                    -0.005952         0.019350           -0.308           0.758
L24.vix                    -0.000030         0.000629           -0.048           0.961
L24.rec_ind                 0.039327         0.024790            1.586           0.113
L24.1yr-ffr                 0.001065         0.003561            0.299           0.765
L25.yieldsp_diff           -0.046508         0.018556           -2.506           0.012
L25.termpr_diff             0.180640         0.067495            2.676           0.007
L25.forward1yr_diff        -0.063505         0.029851           -2.127           0.033
L25.ted                     0.036793         0.019348            1.902           0.057
L25.vix                    -0.000422         0.000628           -0.673           0.501
L25.rec_ind                -0.009884         0.024788           -0.399           0.690
L25.1yr-ffr                -0.005134         0.003564           -1.440           0.150
L26.yieldsp_diff            0.006031         0.018563            0.325           0.745
L26.termpr_diff            -0.094013         0.067539           -1.392           0.164
L26.forward1yr_diff         0.037847         0.029845            1.268           0.205
L26.ted                    -0.003960         0.019352           -0.205           0.838
L26.vix                    -0.000280         0.000628           -0.446           0.656
L26.rec_ind                -0.009648         0.024770           -0.390           0.697
L26.1yr-ffr                 0.003638         0.003562            1.021           0.307
L27.yieldsp_diff            0.010581         0.018556            0.570           0.569
L27.termpr_diff             0.021158         0.067579            0.313           0.754
L27.forward1yr_diff        -0.038063         0.029827           -1.276           0.202
L27.ted                     0.020016         0.019336            1.035           0.301
L27.vix                    -0.000147         0.000628           -0.234           0.815
L27.rec_ind                -0.001747         0.024760           -0.071           0.944
L27.1yr-ffr                -0.000406         0.003562           -0.114           0.909
L28.yieldsp_diff           -0.017444         0.018551           -0.940           0.347
L28.termpr_diff             0.037810         0.067586            0.559           0.576
L28.forward1yr_diff         0.012959         0.029829            0.434           0.664
L28.ted                     0.013260         0.019331            0.686           0.493
L28.vix                     0.000032         0.000627            0.052           0.959
L28.rec_ind                 0.019308         0.024746            0.780           0.435
L28.1yr-ffr                -0.000583         0.003559           -0.164           0.870
L29.yieldsp_diff           -0.007975         0.018551           -0.430           0.667
L29.termpr_diff             0.039129         0.067589            0.579           0.563
L29.forward1yr_diff        -0.046004         0.029822           -1.543           0.123
L29.ted                    -0.023859         0.019319           -1.235           0.217
L29.vix                     0.000261         0.000627            0.416           0.677
L29.rec_ind                -0.004926         0.024743           -0.199           0.842
L29.1yr-ffr                -0.001126         0.003577           -0.315           0.753
L30.yieldsp_diff            0.000275         0.018520            0.015           0.988
L30.termpr_diff            -0.067163         0.067580           -0.994           0.320
L30.forward1yr_diff         0.006582         0.029831            0.221           0.825
L30.ted                    -0.066842         0.019314           -3.461           0.001
L30.vix                    -0.000047         0.000626           -0.075           0.940
L30.rec_ind                 0.031708         0.024738            1.282           0.200
L30.1yr-ffr                -0.002140         0.003583           -0.597           0.550
L31.yieldsp_diff           -0.015644         0.018492           -0.846           0.398
L31.termpr_diff             0.046387         0.067584            0.686           0.492
L31.forward1yr_diff        -0.007902         0.029838           -0.265           0.791
L31.ted                    -0.003369         0.019302           -0.175           0.861
L31.vix                    -0.000181         0.000624           -0.290           0.772
L31.rec_ind                -0.056857         0.024742           -2.298           0.022
L31.1yr-ffr                 0.001740         0.003579            0.486           0.627
L32.yieldsp_diff            0.003146         0.018493            0.170           0.865
L32.termpr_diff            -0.085928         0.067522           -1.273           0.203
L32.forward1yr_diff         0.067371         0.029817            2.259           0.024
L32.ted                     0.028646         0.019231            1.490           0.136
L32.vix                     0.000833         0.000623            1.338           0.181
L32.rec_ind                 0.044516         0.024772            1.797           0.072
L32.1yr-ffr                -0.001369         0.003548           -0.386           0.700
L33.yieldsp_diff            0.015923         0.018501            0.861           0.389
L33.termpr_diff            -0.059691         0.067550           -0.884           0.377
L33.forward1yr_diff         0.031927         0.029829            1.070           0.284
L33.ted                     0.005084         0.019245            0.264           0.792
L33.vix                    -0.001389         0.000623           -2.232           0.026
L33.rec_ind                -0.014908         0.024768           -0.602           0.547
L33.1yr-ffr                -0.001536         0.003545           -0.433           0.665
L34.yieldsp_diff            0.018325         0.018500            0.991           0.322
L34.termpr_diff             0.021562         0.067511            0.319           0.749
L34.forward1yr_diff        -0.022932         0.029811           -0.769           0.442
L34.ted                    -0.032160         0.019257           -1.670           0.095
L34.vix                     0.001936         0.000623            3.110           0.002
L34.rec_ind                -0.027471         0.024759           -1.110           0.267
L34.1yr-ffr                -0.003992         0.003542           -1.127           0.260
L35.yieldsp_diff            0.016962         0.018493            0.917           0.359
L35.termpr_diff            -0.047389         0.067538           -0.702           0.483
L35.forward1yr_diff         0.047251         0.029814            1.585           0.113
L35.ted                     0.018476         0.019262            0.959           0.337
L35.vix                    -0.001457         0.000623           -2.341           0.019
L35.rec_ind                 0.030986         0.024745            1.252           0.211
L35.1yr-ffr                 0.002803         0.003544            0.791           0.429
L36.yieldsp_diff           -0.018848         0.018492           -1.019           0.308
L36.termpr_diff             0.000944         0.067535            0.014           0.989
L36.forward1yr_diff         0.013189         0.029825            0.442           0.658
L36.ted                     0.030451         0.019245            1.582           0.114
L36.vix                     0.000534         0.000622            0.858           0.391
L36.rec_ind                -0.000168         0.024728           -0.007           0.995
L36.1yr-ffr                 0.000811         0.003540            0.229           0.819
L37.yieldsp_diff            0.013067         0.018490            0.707           0.480
L37.termpr_diff            -0.069437         0.067517           -1.028           0.304
L37.forward1yr_diff         0.048489         0.029817            1.626           0.104
L37.ted                    -0.022535         0.019231           -1.172           0.241
L37.vix                     0.000241         0.000622            0.387           0.699
L37.rec_ind                -0.014252         0.024729           -0.576           0.564
L37.1yr-ffr                -0.000071         0.003541           -0.020           0.984
L38.yieldsp_diff           -0.030083         0.018445           -1.631           0.103
L38.termpr_diff             0.129022         0.067386            1.915           0.056
L38.forward1yr_diff        -0.046706         0.029793           -1.568           0.117
L38.ted                    -0.001243         0.019234           -0.065           0.948
L38.vix                    -0.000482         0.000622           -0.775           0.438
L38.rec_ind                -0.011088         0.024724           -0.448           0.654
L38.1yr-ffr                 0.005684         0.003541            1.605           0.109
L39.yieldsp_diff           -0.039074         0.018446           -2.118           0.034
L39.termpr_diff             0.079908         0.067334            1.187           0.235
L39.forward1yr_diff         0.003844         0.029774            0.129           0.897
L39.ted                     0.023275         0.019209            1.212           0.226
L39.vix                     0.000787         0.000621            1.268           0.205
L39.rec_ind                -0.001138         0.024717           -0.046           0.963
L39.1yr-ffr                -0.000039         0.003536           -0.011           0.991
L40.yieldsp_diff           -0.005233         0.018397           -0.284           0.776
L40.termpr_diff            -0.007378         0.067225           -0.110           0.913
L40.forward1yr_diff        -0.021325         0.029727           -0.717           0.473
L40.ted                    -0.006194         0.019201           -0.323           0.747
L40.vix                     0.000053         0.000621            0.085           0.932
L40.rec_ind                 0.012913         0.024716            0.522           0.601
L40.1yr-ffr                -0.003496         0.003532           -0.990           0.322
L41.yieldsp_diff           -0.058147         0.018357           -3.168           0.002
L41.termpr_diff             0.108080         0.067151            1.610           0.108
L41.forward1yr_diff        -0.009734         0.029720           -0.328           0.743
L41.ted                     0.047989         0.019189            2.501           0.012
L41.vix                    -0.001178         0.000620           -1.899           0.058
L41.rec_ind                 0.003865         0.024722            0.156           0.876
L41.1yr-ffr                -0.000184         0.003531           -0.052           0.959
L42.yieldsp_diff            0.002767         0.018211            0.152           0.879
L42.termpr_diff             0.128207         0.066712            1.922           0.055
L42.forward1yr_diff        -0.083364         0.029646           -2.812           0.005
L42.ted                     0.009175         0.019181            0.478           0.632
L42.vix                     0.001029         0.000620            1.659           0.097
L42.rec_ind                 0.021879         0.024719            0.885           0.376
L42.1yr-ffr                 0.000715         0.003528            0.203           0.839
L43.yieldsp_diff            0.022064         0.017933            1.230           0.219
L43.termpr_diff            -0.128065         0.065965           -1.941           0.052
L43.forward1yr_diff         0.022132         0.029464            0.751           0.453
L43.ted                    -0.075480         0.019169           -3.938           0.000
L43.vix                    -0.000724         0.000615           -1.177           0.239
L43.rec_ind                -0.014828         0.024715           -0.600           0.549
L43.1yr-ffr                -0.003123         0.003535           -0.883           0.377
L44.yieldsp_diff           -0.032695         0.017896           -1.827           0.068
L44.termpr_diff             0.094022         0.065850            1.428           0.153
L44.forward1yr_diff        -0.000263         0.029292           -0.009           0.993
L44.ted                     0.015078         0.019141            0.788           0.431
L44.vix                     0.000537         0.000583            0.920           0.357
L44.rec_ind                -0.014768         0.024712           -0.598           0.550
L44.1yr-ffr                 0.005326         0.003529            1.509           0.131
L45.yieldsp_diff           -0.043713         0.013485           -3.242           0.001
L45.termpr_diff             0.076093         0.055335            1.375           0.169
L45.forward1yr_diff        -0.001584         0.025290           -0.063           0.950
L45.ted                     0.012112         0.013522            0.896           0.370
L45.vix                    -0.000573         0.000364           -1.576           0.115
L45.rec_ind                 0.012815         0.017499            0.732           0.464
L45.1yr-ffr                -0.002741         0.003001           -0.913           0.361
======================================================================================

Results for equation vix
======================================================================================
                         coefficient       std. error           t-stat            prob
--------------------------------------------------------------------------------------
const                       0.163161         0.035805            4.557           0.000
L1.yieldsp_diff            -0.304946         0.486340           -0.627           0.531
L1.termpr_diff              1.629512         1.702205            0.957           0.338
L1.forward1yr_diff         -0.607547         0.715038           -0.850           0.396
L1.ted                      0.522019         0.363596            1.436           0.151
L1.vix                      1.257922         0.009962          126.274           0.000
L1.rec_ind                 -0.279816         0.468292           -0.598           0.550
L1.1yr-ffr                  0.114309         0.081592            1.401           0.161
L2.yieldsp_diff            -1.905469         0.488471           -3.901           0.000
L2.termpr_diff              3.387432         1.800717            1.881           0.060
L2.forward1yr_diff         -0.887440         0.791787           -1.121           0.262
L2.ted                      0.880691         0.513281            1.716           0.086
L2.vix                     -0.535222         0.015995          -33.462           0.000
L2.rec_ind                  2.136920         0.661500            3.230           0.001
L2.1yr-ffr                 -0.056296         0.095479           -0.590           0.555
L3.yieldsp_diff            -0.797658         0.490004           -1.628           0.104
L3.termpr_diff              1.296553         1.808016            0.717           0.473
L3.forward1yr_diff         -0.335765         0.796342           -0.422           0.673
L3.ted                     -0.813527         0.513141           -1.585           0.113
L3.vix                      0.228332         0.016854           13.548           0.000
L3.rec_ind                 -0.772519         0.662059           -1.167           0.243
L3.1yr-ffr                  0.007213         0.095479            0.076           0.940
L4.yieldsp_diff            -0.516551         0.495858           -1.042           0.298
L4.termpr_diff             -1.115097         1.812422           -0.615           0.538
L4.forward1yr_diff         -0.100088         0.799456           -0.125           0.900
L4.ted                     -0.569522         0.513177           -1.110           0.267
L4.vix                     -0.013254         0.016999           -0.780           0.436
L4.rec_ind                 -0.992639         0.662266           -1.499           0.134
L4.1yr-ffr                  0.134069         0.095308            1.407           0.160
L5.yieldsp_diff            -0.281016         0.497784           -0.565           0.572
L5.termpr_diff              2.588193         1.814573            1.426           0.154
L5.forward1yr_diff         -1.791855         0.800390           -2.239           0.025
L5.ted                      0.728121         0.513881            1.417           0.157
L5.vix                     -0.016837         0.016985           -0.991           0.322
L5.rec_ind                  0.480306         0.662658            0.725           0.469
L5.1yr-ffr                  0.073779         0.095297            0.774           0.439
L6.yieldsp_diff            -0.576048         0.499736           -1.153           0.249
L6.termpr_diff             -0.475849         1.814969           -0.262           0.793
L6.forward1yr_diff          0.105512         0.800390            0.132           0.895
L6.ted                     -0.650878         0.514204           -1.266           0.206
L6.vix                      0.023735         0.016968            1.399           0.162
L6.rec_ind                 -0.641279         0.662478           -0.968           0.333
L6.1yr-ffr                  0.055175         0.095345            0.579           0.563
L7.yieldsp_diff            -1.317437         0.500024           -2.635           0.008
L7.termpr_diff              5.701926         1.815029            3.142           0.002
L7.forward1yr_diff         -1.801448         0.800554           -2.250           0.024
L7.ted                      0.463215         0.513805            0.902           0.367
L7.vix                      0.017473         0.016949            1.031           0.303
L7.rec_ind                 -0.267731         0.662545           -0.404           0.686
L7.1yr-ffr                 -0.107353         0.095521           -1.124           0.261
L8.yieldsp_diff            -0.252720         0.500766           -0.505           0.614
L8.termpr_diff             -4.074763         1.819235           -2.240           0.025
L8.forward1yr_diff          1.343302         0.801786            1.675           0.094
L8.ted                     -0.771796         0.514151           -1.501           0.133
L8.vix                     -0.001411         0.016947           -0.083           0.934
L8.rec_ind                 -0.963477         0.662497           -1.454           0.146
L8.1yr-ffr                  0.059431         0.095725            0.621           0.535
L9.yieldsp_diff            -1.213699         0.500249           -2.426           0.015
L9.termpr_diff              3.502925         1.820446            1.924           0.054
L9.forward1yr_diff         -1.240799         0.802415           -1.546           0.122
L9.ted                      0.582839         0.514743            1.132           0.258
L9.vix                     -0.011963         0.016939           -0.706           0.480
L9.rec_ind                  1.456228         0.662850            2.197           0.028
L9.1yr-ffr                  0.021180         0.095668            0.221           0.825
L10.yieldsp_diff           -0.354493         0.500420           -0.708           0.479
L10.termpr_diff            -1.776196         1.820493           -0.976           0.329
L10.forward1yr_diff         1.067623         0.802687            1.330           0.183
L10.ted                     0.700072         0.515113            1.359           0.174
L10.vix                     0.013650         0.016932            0.806           0.420
L10.rec_ind                 0.163956         0.663463            0.247           0.805
L10.1yr-ffr                -0.173506         0.095709           -1.813           0.070
L11.yieldsp_diff            0.751645         0.500005            1.503           0.133
L11.termpr_diff            -0.200001         1.819731           -0.110           0.912
L11.forward1yr_diff        -0.948105         0.803135           -1.181           0.238
L11.ted                    -1.259690         0.515552           -2.443           0.015
L11.vix                    -0.000752         0.016921           -0.044           0.965
L11.rec_ind                -0.900637         0.663400           -1.358           0.175
L11.1yr-ffr                -0.034873         0.095788           -0.364           0.716
L12.yieldsp_diff           -0.552379         0.500144           -1.104           0.269
L12.termpr_diff             0.972572         1.821053            0.534           0.593
L12.forward1yr_diff         0.330007         0.803432            0.411           0.681
L12.ted                     0.721229         0.515812            1.398           0.162
L12.vix                     0.018253         0.016910            1.079           0.280
L12.rec_ind                 0.474063         0.663361            0.715           0.475
L12.1yr-ffr                -0.014195         0.095715           -0.148           0.882
L13.yieldsp_diff           -0.375223         0.500047           -0.750           0.453
L13.termpr_diff             0.314408         1.821028            0.173           0.863
L13.forward1yr_diff         0.046274         0.803536            0.058           0.954
L13.ted                    -0.088234         0.516005           -0.171           0.864
L13.vix                    -0.038461         0.016906           -2.275           0.023
L13.rec_ind                 0.558356         0.663321            0.842           0.400
L13.1yr-ffr                 0.108896         0.095621            1.139           0.255
L14.yieldsp_diff           -1.534210         0.499452           -3.072           0.002
L14.termpr_diff             5.394126         1.819270            2.965           0.003
L14.forward1yr_diff        -1.492035         0.803659           -1.857           0.063
L14.ted                     0.998550         0.516369            1.934           0.053
L14.vix                     0.125087         0.016901            7.401           0.000
L14.rec_ind                -0.380514         0.662680           -0.574           0.566
L14.1yr-ffr                -0.068849         0.095622           -0.720           0.472
L15.yieldsp_diff            0.341672         0.499669            0.684           0.494
L15.termpr_diff            -4.912237         1.819430           -2.700           0.007
L15.forward1yr_diff         1.322741         0.803771            1.646           0.100
L15.ted                    -1.322827         0.518742           -2.550           0.011
L15.vix                    -0.126130         0.016947           -7.443           0.000
L15.rec_ind                 1.064110         0.662009            1.607           0.108
L15.1yr-ffr                -0.081406         0.096467           -0.844           0.399
L16.yieldsp_diff           -1.067689         0.499463           -2.138           0.033
L16.termpr_diff             3.961652         1.819563            2.177           0.029
L16.forward1yr_diff        -0.411258         0.803666           -0.512           0.609
L16.ted                     1.567423         0.519290            3.018           0.003
L16.vix                     0.052417         0.016980            3.087           0.002
L16.rec_ind                -0.737801         0.662021           -1.114           0.265
L16.1yr-ffr                 0.316442         0.096626            3.275           0.001
L17.yieldsp_diff           -1.607751         0.499399           -3.219           0.001
L17.termpr_diff             4.233354         1.819516            2.327           0.020
L17.forward1yr_diff        -0.459602         0.803868           -0.572           0.567
L17.ted                    -0.168371         0.519413           -0.324           0.746
L17.vix                    -0.009781         0.016974           -0.576           0.564
L17.rec_ind                -1.167215         0.661953           -1.763           0.078
L17.1yr-ffr                -0.225676         0.096508           -2.338           0.019
L18.yieldsp_diff            0.471545         0.499789            0.943           0.345
L18.termpr_diff            -0.704051         1.818366           -0.387           0.699
L18.forward1yr_diff        -1.068784         0.803607           -1.330           0.184
L18.ted                    -2.098891         0.519335           -4.041           0.000
L18.vix                    -0.009067         0.016959           -0.535           0.593
L18.rec_ind                 1.225807         0.662746            1.850           0.064
L18.1yr-ffr                -0.085695         0.095982           -0.893           0.372
L19.yieldsp_diff           -0.235872         0.499951           -0.472           0.637
L19.termpr_diff            -1.663825         1.817900           -0.915           0.360
L19.forward1yr_diff         1.215334         0.803422            1.513           0.130
L19.ted                     0.344267         0.519930            0.662           0.508
L19.vix                     0.017158         0.016945            1.013           0.311
L19.rec_ind                 0.064543         0.663389            0.097           0.922
L19.1yr-ffr                 0.040835         0.095897            0.426           0.670
L20.yieldsp_diff           -0.994152         0.499610           -1.990           0.047
L20.termpr_diff             1.869681         1.818355            1.028           0.304
L20.forward1yr_diff        -0.688066         0.803282           -0.857           0.392
L20.ted                     1.082213         0.520026            2.081           0.037
L20.vix                    -0.028510         0.016942           -1.683           0.092
L20.rec_ind                 0.588538         0.664773            0.885           0.376
L20.1yr-ffr                 0.043249         0.095872            0.451           0.652
L21.yieldsp_diff            0.612750         0.499526            1.227           0.220
L21.termpr_diff            -2.641470         1.818170           -1.453           0.146
L21.forward1yr_diff         0.578730         0.803444            0.720           0.471
L21.ted                    -1.091509         0.520251           -2.098           0.036
L21.vix                     0.024461         0.016939            1.444           0.149
L21.rec_ind                -0.252236         0.664671           -0.379           0.704
L21.1yr-ffr                 0.141612         0.095906            1.477           0.140
L22.yieldsp_diff           -0.340587         0.499887           -0.681           0.496
L22.termpr_diff             1.869792         1.819651            1.028           0.304
L22.forward1yr_diff        -0.045514         0.804059           -0.057           0.955
L22.ted                     1.586857         0.520450            3.049           0.002
L22.vix                     0.000912         0.016936            0.054           0.957
L22.rec_ind                 0.072803         0.664986            0.109           0.913
L22.1yr-ffr                -0.131247         0.095886           -1.369           0.171
L23.yieldsp_diff            0.658028         0.499502            1.317           0.188
L23.termpr_diff            -2.116726         1.819256           -1.164           0.245
L23.forward1yr_diff        -0.690998         0.803704           -0.860           0.390
L23.ted                    -1.038159         0.520387           -1.995           0.046
L23.vix                    -0.012209         0.016926           -0.721           0.471
L23.rec_ind                -1.493431         0.666016           -2.242           0.025
L23.1yr-ffr                 0.116454         0.095891            1.214           0.225
L24.yieldsp_diff            0.607715         0.499355            1.217           0.224
L24.termpr_diff             0.665569         1.817977            0.366           0.714
L24.forward1yr_diff        -1.036401         0.803182           -1.290           0.197
L24.ted                    -0.347678         0.520503           -0.668           0.504
L24.vix                     0.017060         0.016913            1.009           0.313
L24.rec_ind                 0.531161         0.666840            0.797           0.426
L24.1yr-ffr                 0.026062         0.095795            0.272           0.786
L25.yieldsp_diff            0.732277         0.499143            1.467           0.142
L25.termpr_diff            -2.625358         1.815570           -1.446           0.148
L25.forward1yr_diff         0.110901         0.802976            0.138           0.890
L25.ted                    -1.141260         0.520451           -2.193           0.028
L25.vix                    -0.014957         0.016897           -0.885           0.376
L25.rec_ind                 0.214073         0.666789            0.321           0.748
L25.1yr-ffr                -0.036136         0.095863           -0.377           0.706
L26.yieldsp_diff           -0.699163         0.499342           -1.400           0.161
L26.termpr_diff             3.450485         1.816757            1.899           0.058
L26.forward1yr_diff        -0.476426         0.802800           -0.593           0.553
L26.ted                     0.995150         0.520550            1.912           0.056
L26.vix                     0.001019         0.016886            0.060           0.952
L26.rec_ind                -0.525315         0.666293           -0.788           0.430
L26.1yr-ffr                 0.105273         0.095821            1.099           0.272
L27.yieldsp_diff           -0.711870         0.499131           -1.426           0.154
L27.termpr_diff             0.008354         1.817823            0.005           0.996
L27.forward1yr_diff        -0.042206         0.802326           -0.053           0.958
L27.ted                     1.079922         0.520123            2.076           0.038
L27.vix                     0.029087         0.016882            1.723           0.085
L27.rec_ind                 0.132957         0.666015            0.200           0.842
L27.1yr-ffr                 0.017813         0.095802            0.186           0.852
L28.yieldsp_diff            0.438713         0.499002            0.879           0.379
L28.termpr_diff             1.699981         1.818009            0.935           0.350
L28.forward1yr_diff        -0.761858         0.802365           -0.950           0.342
L28.ted                    -1.203826         0.519977           -2.315           0.021
L28.vix                    -0.033393         0.016872           -1.979           0.048
L28.rec_ind                -0.042101         0.665648           -0.063           0.950
L28.1yr-ffr                -0.037963         0.095745           -0.396           0.692
L29.yieldsp_diff           -0.425785         0.499000           -0.853           0.394
L29.termpr_diff            -2.016359         1.818084           -1.109           0.267
L29.forward1yr_diff         1.330594         0.802199            1.659           0.097
L29.ted                     0.339461         0.519652            0.653           0.514
L29.vix                     0.036232         0.016863            2.149           0.032
L29.rec_ind                 0.415637         0.665568            0.624           0.532
L29.1yr-ffr                 0.091624         0.096218            0.952           0.341
L30.yieldsp_diff           -0.193012         0.498174           -0.387           0.698
L30.termpr_diff             3.483952         1.817851            1.917           0.055
L30.forward1yr_diff        -2.309675         0.802433           -2.878           0.004
L30.ted                    -0.327255         0.519544           -0.630           0.529
L30.vix                    -0.027505         0.016851           -1.632           0.103
L30.rec_ind                -0.738811         0.665438           -1.110           0.267
L30.1yr-ffr                -0.198817         0.096379           -2.063           0.039
L31.yieldsp_diff            0.342103         0.497423            0.688           0.492
L31.termpr_diff            -2.839293         1.817955           -1.562           0.118
L31.forward1yr_diff         1.866926         0.802615            2.326           0.020
L31.ted                     0.907225         0.519197            1.747           0.081
L31.vix                    -0.006247         0.016792           -0.372           0.710
L31.rec_ind                 0.639942         0.665528            0.962           0.336
L31.1yr-ffr                 0.040641         0.096269            0.422           0.673
L32.yieldsp_diff           -0.125206         0.497446           -0.252           0.801
L32.termpr_diff             2.689486         1.816275            1.481           0.139
L32.forward1yr_diff        -2.391598         0.802060           -2.982           0.003
L32.ted                    -1.120778         0.517288           -2.167           0.030
L32.vix                    -0.009490         0.016746           -0.567           0.571
L32.rec_ind                 0.557060         0.666338            0.836           0.403
L32.1yr-ffr                 0.017551         0.095427            0.184           0.854
L33.yieldsp_diff           -1.403929         0.497664           -2.821           0.005
L33.termpr_diff             0.343161         1.817041            0.189           0.850
L33.forward1yr_diff         1.104576         0.802370            1.377           0.169
L33.ted                     0.465872         0.517675            0.900           0.368
L33.vix                     0.007375         0.016746            0.440           0.660
L33.rec_ind                -0.700425         0.666234           -1.051           0.293
L33.1yr-ffr                 0.052606         0.095370            0.552           0.581
L34.yieldsp_diff           -0.963244         0.497625           -1.936           0.053
L34.termpr_diff             4.005212         1.815982            2.206           0.027
L34.forward1yr_diff        -1.680935         0.801888           -2.096           0.036
L34.ted                    -0.496035         0.517997           -0.958           0.338
L34.vix                     0.013002         0.016748            0.776           0.438
L34.rec_ind                 0.563968         0.665996            0.847           0.397
L34.1yr-ffr                -0.072093         0.095287           -0.757           0.449
L35.yieldsp_diff            0.120918         0.497458            0.243           0.808
L35.termpr_diff            -1.060818         1.816711           -0.584           0.559
L35.forward1yr_diff         1.330888         0.801960            1.660           0.097
L35.ted                     0.668935         0.518143            1.291           0.197
L35.vix                     0.021088         0.016746            1.259           0.208
L35.rec_ind                -0.059791         0.665632           -0.090           0.928
L35.1yr-ffr                -0.206157         0.095322           -2.163           0.031
L36.yieldsp_diff           -1.112683         0.497417           -2.237           0.025
L36.termpr_diff             4.157035         1.816649            2.288           0.022
L36.forward1yr_diff        -1.262338         0.802269           -1.573           0.116
L36.ted                    -0.639641         0.517668           -1.236           0.217
L36.vix                    -0.025857         0.016733           -1.545           0.122
L36.rec_ind                 0.178300         0.665160            0.268           0.789
L36.1yr-ffr                 0.177568         0.095233            1.865           0.062
L37.yieldsp_diff           -1.530043         0.497356           -3.076           0.002
L37.termpr_diff             1.832446         1.816160            1.009           0.313
L37.forward1yr_diff         0.974096         0.802048            1.215           0.225
L37.ted                     2.063818         0.517295            3.990           0.000
L37.vix                     0.003833         0.016729            0.229           0.819
L37.rec_ind                -0.977213         0.665188           -1.469           0.142
L37.1yr-ffr                -0.115369         0.095239           -1.211           0.226
L38.yieldsp_diff           -0.943898         0.496159           -1.902           0.057
L38.termpr_diff             1.900078         1.812631            1.048           0.295
L38.forward1yr_diff        -0.815400         0.801402           -1.017           0.309
L38.ted                    -0.445140         0.517383           -0.860           0.390
L38.vix                     0.017345         0.016719            1.037           0.300
L38.rec_ind                 0.986094         0.665061            1.483           0.138
L38.1yr-ffr                -0.097576         0.095263           -1.024           0.306
L39.yieldsp_diff            0.240932         0.496182            0.486           0.627
L39.termpr_diff            -0.158906         1.811237           -0.088           0.930
L39.forward1yr_diff        -1.513138         0.800903           -1.889           0.059
L39.ted                    -0.945211         0.516717           -1.829           0.067
L39.vix                    -0.020218         0.016703           -1.210           0.226
L39.rec_ind                -0.912523         0.664869           -1.372           0.170
L39.1yr-ffr                -0.089797         0.095125           -0.944           0.345
L40.yieldsp_diff            0.047862         0.494862            0.097           0.923
L40.termpr_diff            -2.741427         1.808301           -1.516           0.130
L40.forward1yr_diff         2.327996         0.799623            2.911           0.004
L40.ted                     0.086364         0.516487            0.167           0.867
L40.vix                    -0.000276         0.016704           -0.017           0.987
L40.rec_ind                -0.017126         0.664853           -0.026           0.979
L40.1yr-ffr                 0.092557         0.095002            0.974           0.330
L41.yieldsp_diff           -0.953509         0.493795           -1.931           0.053
L41.termpr_diff             5.075286         1.806296            2.810           0.005
L41.forward1yr_diff        -3.061664         0.799456           -3.830           0.000
L41.ted                     0.432020         0.516174            0.837           0.403
L41.vix                    -0.010701         0.016689           -0.641           0.521
L41.rec_ind                -0.132220         0.665002           -0.199           0.842
L41.1yr-ffr                 0.078974         0.094983            0.831           0.406
L42.yieldsp_diff           -0.179920         0.489872           -0.367           0.713
L42.termpr_diff            -2.088382         1.794493           -1.164           0.245
L42.forward1yr_diff         1.708865         0.797445            2.143           0.032
L42.ted                    -0.015654         0.515961           -0.030           0.976
L42.vix                     0.009861         0.016680            0.591           0.554
L42.rec_ind                 0.474391         0.664916            0.713           0.476
L42.1yr-ffr                -0.180374         0.094897           -1.901           0.057
L43.yieldsp_diff           -0.503389         0.482391           -1.044           0.297
L43.termpr_diff             1.939922         1.774407            1.093           0.274
L43.forward1yr_diff        -1.193618         0.792564           -1.506           0.132
L43.ted                    -0.854904         0.515630           -1.658           0.097
L43.vix                    -0.004220         0.016536           -0.255           0.799
L43.rec_ind                -0.250496         0.664814           -0.377           0.706
L43.1yr-ffr                 0.089051         0.095099            0.936           0.349
L44.yieldsp_diff           -0.252056         0.481401           -0.524           0.601
L44.termpr_diff             0.115616         1.771317            0.065           0.948
L44.forward1yr_diff         0.142477         0.787918            0.181           0.857
L44.ted                    -0.483539         0.514888           -0.939           0.348
L44.vix                     0.019778         0.015693            1.260           0.208
L44.rec_ind                 0.607196         0.664731            0.913           0.361
L44.1yr-ffr                -0.009844         0.094930           -0.104           0.917
L45.yieldsp_diff           -0.562162         0.362737           -1.550           0.121
L45.termpr_diff             1.870527         1.488461            1.257           0.209
L45.forward1yr_diff        -0.353803         0.680283           -0.520           0.603
L45.ted                     0.739954         0.363733            2.034           0.042
L45.vix                    -0.009176         0.009790           -0.937           0.349
L45.rec_ind                -0.288202         0.470714           -0.612           0.540
L45.1yr-ffr                 0.004456         0.080723            0.055           0.956
======================================================================================

Results for equation rec_ind
======================================================================================
                         coefficient       std. error           t-stat            prob
--------------------------------------------------------------------------------------
const                      -0.000654         0.000753           -0.869           0.385
L1.yieldsp_diff             0.011436         0.010231            1.118           0.264
L1.termpr_diff             -0.016770         0.035809           -0.468           0.640
L1.forward1yr_diff         -0.000528         0.015042           -0.035           0.972
L1.ted                     -0.008437         0.007649           -1.103           0.270
L1.vix                      0.000142         0.000210            0.678           0.498
L1.rec_ind                  0.997073         0.009851          101.211           0.000
L1.1yr-ffr                  0.001387         0.001716            0.808           0.419
L2.yieldsp_diff             0.003288         0.010276            0.320           0.749
L2.termpr_diff             -0.009223         0.037882           -0.243           0.808
L2.forward1yr_diff          0.014074         0.016657            0.845           0.398
L2.ted                      0.006863         0.010798            0.636           0.525
L2.vix                      0.000004         0.000336            0.011           0.991
L2.rec_ind                  0.002602         0.013916            0.187           0.852
L2.1yr-ffr                  0.002279         0.002009            1.134           0.257
L3.yieldsp_diff            -0.021588         0.010308           -2.094           0.036
L3.termpr_diff              0.065092         0.038035            1.711           0.087
L3.forward1yr_diff         -0.027137         0.016753           -1.620           0.105
L3.ted                      0.006590         0.010795            0.611           0.542
L3.vix                     -0.000178         0.000355           -0.502           0.616
L3.rec_ind                 -0.007014         0.013928           -0.504           0.615
L3.1yr-ffr                  0.005132         0.002009            2.555           0.011
L4.yieldsp_diff             0.001686         0.010431            0.162           0.872
L4.termpr_diff             -0.018230         0.038128           -0.478           0.633
L4.forward1yr_diff         -0.005318         0.016818           -0.316           0.752
L4.ted                     -0.017226         0.010796           -1.596           0.111
L4.vix                      0.000209         0.000358            0.584           0.559
L4.rec_ind                  0.001937         0.013932            0.139           0.889
L4.1yr-ffr                 -0.004893         0.002005           -2.440           0.015
L5.yieldsp_diff             0.006854         0.010472            0.655           0.513
L5.termpr_diff              0.013901         0.038173            0.364           0.716
L5.forward1yr_diff         -0.008069         0.016838           -0.479           0.632
L5.ted                      0.004122         0.010811            0.381           0.703
L5.vix                     -0.000048         0.000357           -0.133           0.894
L5.rec_ind                  0.000411         0.013940            0.029           0.977
L5.1yr-ffr                  0.000050         0.002005            0.025           0.980
L6.yieldsp_diff            -0.002728         0.010513           -0.260           0.795
L6.termpr_diff              0.027546         0.038182            0.721           0.471
L6.forward1yr_diff         -0.007830         0.016838           -0.465           0.642
L6.ted                      0.005035         0.010817            0.466           0.642
L6.vix                      0.000036         0.000357            0.101           0.919
L6.rec_ind                  0.002950         0.013937            0.212           0.832
L6.1yr-ffr                  0.002186         0.002006            1.090           0.276
L7.yieldsp_diff            -0.005224         0.010519           -0.497           0.619
L7.termpr_diff             -0.056864         0.038183           -1.489           0.136
L7.forward1yr_diff          0.035063         0.016841            2.082           0.037
L7.ted                      0.022095         0.010809            2.044           0.041
L7.vix                     -0.000153         0.000357           -0.428           0.669
L7.rec_ind                 -0.003323         0.013938           -0.238           0.812
L7.1yr-ffr                 -0.003633         0.002009           -1.808           0.071
L8.yieldsp_diff            -0.016338         0.010535           -1.551           0.121
L8.termpr_diff              0.095334         0.038271            2.491           0.013
L8.forward1yr_diff         -0.033532         0.016867           -1.988           0.047
L8.ted                     -0.015310         0.010816           -1.415           0.157
L8.vix                     -0.000071         0.000357           -0.199           0.842
L8.rec_ind                 -0.000673         0.013937           -0.048           0.961
L8.1yr-ffr                  0.002869         0.002014            1.425           0.154
L9.yieldsp_diff            -0.000632         0.010524           -0.060           0.952
L9.termpr_diff             -0.014338         0.038297           -0.374           0.708
L9.forward1yr_diff          0.000102         0.016880            0.006           0.995
L9.ted                     -0.003529         0.010829           -0.326           0.744
L9.vix                      0.000056         0.000356            0.156           0.876
L9.rec_ind                  0.002652         0.013944            0.190           0.849
L9.1yr-ffr                  0.000079         0.002013            0.039           0.969
L10.yieldsp_diff           -0.005547         0.010527           -0.527           0.598
L10.termpr_diff            -0.009258         0.038298           -0.242           0.809
L10.forward1yr_diff        -0.004355         0.016886           -0.258           0.796
L10.ted                    -0.009504         0.010836           -0.877           0.380
L10.vix                     0.000226         0.000356            0.635           0.525
L10.rec_ind                 0.004167         0.013957            0.299           0.765
L10.1yr-ffr                 0.000312         0.002013            0.155           0.877
L11.yieldsp_diff            0.008997         0.010519            0.855           0.392
L11.termpr_diff            -0.038387         0.038282           -1.003           0.316
L11.forward1yr_diff         0.029687         0.016896            1.757           0.079
L11.ted                     0.031678         0.010846            2.921           0.003
L11.vix                    -0.000295         0.000356           -0.828           0.408
L11.rec_ind                -0.006916         0.013956           -0.496           0.620
L11.1yr-ffr                -0.001962         0.002015           -0.974           0.330
L12.yieldsp_diff            0.011878         0.010522            1.129           0.259
L12.termpr_diff             0.030883         0.038310            0.806           0.420
L12.forward1yr_diff        -0.043573         0.016902           -2.578           0.010
L12.ted                    -0.040160         0.010851           -3.701           0.000
L12.vix                     0.000172         0.000356            0.483           0.629
L12.rec_ind                 0.003958         0.013955            0.284           0.777
L12.1yr-ffr                 0.001518         0.002014            0.754           0.451
L13.yieldsp_diff            0.004642         0.010519            0.441           0.659
L13.termpr_diff            -0.041994         0.038309           -1.096           0.273
L13.forward1yr_diff         0.005458         0.016904            0.323           0.747
L13.ted                     0.019494         0.010855            1.796           0.073
L13.vix                    -0.000182         0.000356           -0.511           0.609
L13.rec_ind                 0.000977         0.013954            0.070           0.944
L13.1yr-ffr                 0.001634         0.002012            0.812           0.417
L14.yieldsp_diff            0.016148         0.010507            1.537           0.124
L14.termpr_diff            -0.049589         0.038272           -1.296           0.195
L14.forward1yr_diff         0.020701         0.016907            1.224           0.221
L14.ted                     0.001305         0.010863            0.120           0.904
L14.vix                     0.000262         0.000356            0.737           0.461
L14.rec_ind                -0.002569         0.013941           -0.184           0.854
L14.1yr-ffr                -0.001053         0.002012           -0.523           0.601
L15.yieldsp_diff            0.002105         0.010512            0.200           0.841
L15.termpr_diff             0.023635         0.038275            0.618           0.537
L15.forward1yr_diff        -0.015471         0.016909           -0.915           0.360
L15.ted                    -0.002000         0.010913           -0.183           0.855
L15.vix                    -0.000411         0.000357           -1.152           0.249
L15.rec_ind                -0.003344         0.013927           -0.240           0.810
L15.1yr-ffr                -0.002370         0.002029           -1.168           0.243
L16.yieldsp_diff            0.015890         0.010507            1.512           0.130
L16.termpr_diff            -0.024303         0.038278           -0.635           0.525
L16.forward1yr_diff        -0.027688         0.016907           -1.638           0.101
L16.ted                    -0.018583         0.010924           -1.701           0.089
L16.vix                     0.000491         0.000357            1.373           0.170
L16.rec_ind                 0.002896         0.013927            0.208           0.835
L16.1yr-ffr                 0.000221         0.002033            0.108           0.914
L17.yieldsp_diff            0.007098         0.010506            0.676           0.499
L17.termpr_diff            -0.019777         0.038277           -0.517           0.605
L17.forward1yr_diff        -0.011835         0.016911           -0.700           0.484
L17.ted                     0.003639         0.010927            0.333           0.739
L17.vix                    -0.000524         0.000357           -1.468           0.142
L17.rec_ind                -0.000712         0.013926           -0.051           0.959
L17.1yr-ffr                 0.000093         0.002030            0.046           0.964
L18.yieldsp_diff           -0.003989         0.010514           -0.379           0.704
L18.termpr_diff             0.011975         0.038253            0.313           0.754
L18.forward1yr_diff         0.012697         0.016905            0.751           0.453
L18.ted                     0.021087         0.010925            1.930           0.054
L18.vix                     0.000978         0.000357            2.740           0.006
L18.rec_ind                -0.002306         0.013942           -0.165           0.869
L18.1yr-ffr                -0.000728         0.002019           -0.360           0.719
L19.yieldsp_diff            0.037082         0.010517            3.526           0.000
L19.termpr_diff            -0.055256         0.038243           -1.445           0.149
L19.forward1yr_diff         0.001216         0.016902            0.072           0.943
L19.ted                    -0.036449         0.010938           -3.332           0.001
L19.vix                    -0.000637         0.000356           -1.787           0.074
L19.rec_ind                 0.004146         0.013956            0.297           0.766
L19.1yr-ffr                -0.001287         0.002017           -0.638           0.523
L20.yieldsp_diff            0.012184         0.010510            1.159           0.246
L20.termpr_diff            -0.013479         0.038253           -0.352           0.725
L20.forward1yr_diff         0.013921         0.016899            0.824           0.410
L20.ted                     0.035584         0.010940            3.253           0.001
L20.vix                     0.000060         0.000356            0.168           0.867
L20.rec_ind                -0.003089         0.013985           -0.221           0.825
L20.1yr-ffr                -0.001753         0.002017           -0.869           0.385
L21.yieldsp_diff           -0.019201         0.010509           -1.827           0.068
L21.termpr_diff             0.080255         0.038249            2.098           0.036
L21.forward1yr_diff        -0.050561         0.016902           -2.991           0.003
L21.ted                    -0.001151         0.010945           -0.105           0.916
L21.vix                     0.000105         0.000356            0.294           0.769
L21.rec_ind                 0.001993         0.013983            0.143           0.887
L21.1yr-ffr                 0.004390         0.002018            2.176           0.030
L22.yieldsp_diff            0.006468         0.010516            0.615           0.538
L22.termpr_diff            -0.032882         0.038280           -0.859           0.390
L22.forward1yr_diff         0.016951         0.016915            1.002           0.316
L22.ted                    -0.000885         0.010949           -0.081           0.936
L22.vix                     0.000275         0.000356            0.772           0.440
L22.rec_ind                -0.003213         0.013989           -0.230           0.818
L22.1yr-ffr                -0.002887         0.002017           -1.431           0.152
L23.yieldsp_diff           -0.001617         0.010508           -0.154           0.878
L23.termpr_diff            -0.007963         0.038272           -0.208           0.835
L23.forward1yr_diff         0.004069         0.016908            0.241           0.810
L23.ted                     0.003419         0.010947            0.312           0.755
L23.vix                    -0.000405         0.000356           -1.136           0.256
L23.rec_ind                -0.002514         0.014011           -0.179           0.858
L23.1yr-ffr                -0.000974         0.002017           -0.483           0.629
L24.yieldsp_diff           -0.009878         0.010505           -0.940           0.347
L24.termpr_diff            -0.027472         0.038245           -0.718           0.473
L24.forward1yr_diff         0.035982         0.016897            2.130           0.033
L24.ted                     0.011931         0.010950            1.090           0.276
L24.vix                    -0.000097         0.000356           -0.273           0.785
L24.rec_ind                 0.007432         0.014028            0.530           0.596
L24.1yr-ffr                -0.000923         0.002015           -0.458           0.647
L25.yieldsp_diff           -0.012252         0.010500           -1.167           0.243
L25.termpr_diff             0.037922         0.038194            0.993           0.321
L25.forward1yr_diff        -0.011648         0.016892           -0.690           0.490
L25.ted                    -0.015421         0.010949           -1.409           0.159
L25.vix                     0.000070         0.000355            0.198           0.843
L25.rec_ind                -0.002943         0.014027           -0.210           0.834
L25.1yr-ffr                 0.001002         0.002017            0.497           0.619
L26.yieldsp_diff            0.000395         0.010505            0.038           0.970
L26.termpr_diff             0.015850         0.038219            0.415           0.678
L26.forward1yr_diff        -0.014502         0.016889           -0.859           0.391
L26.ted                    -0.000473         0.010951           -0.043           0.966
L26.vix                    -0.000346         0.000355           -0.973           0.330
L26.rec_ind                 0.000857         0.014017            0.061           0.951
L26.1yr-ffr                 0.001018         0.002016            0.505           0.613
L27.yieldsp_diff            0.007270         0.010500            0.692           0.489
L27.termpr_diff            -0.072153         0.038242           -1.887           0.059
L27.forward1yr_diff         0.025855         0.016879            1.532           0.126
L27.ted                    -0.004261         0.010942           -0.389           0.697
L27.vix                     0.000427         0.000355            1.203           0.229
L27.rec_ind                 0.002323         0.014011            0.166           0.868
L27.1yr-ffr                -0.000468         0.002015           -0.232           0.816
L28.yieldsp_diff            0.002104         0.010497            0.200           0.841
L28.termpr_diff             0.051961         0.038245            1.359           0.174
L28.forward1yr_diff        -0.025236         0.016879           -1.495           0.135
L28.ted                     0.001649         0.010939            0.151           0.880
L28.vix                    -0.000243         0.000355           -0.685           0.493
L28.rec_ind                -0.006001         0.014003           -0.429           0.668
L28.1yr-ffr                -0.000087         0.002014           -0.043           0.966
L29.yieldsp_diff            0.016015         0.010497            1.526           0.127
L29.termpr_diff            -0.087091         0.038247           -2.277           0.023
L29.forward1yr_diff         0.029378         0.016876            1.741           0.082
L29.ted                    -0.003564         0.010932           -0.326           0.744
L29.vix                     0.000230         0.000355            0.649           0.516
L29.rec_ind                 0.004625         0.014002            0.330           0.741
L29.1yr-ffr                -0.001326         0.002024           -0.655           0.512
L30.yieldsp_diff            0.001459         0.010480            0.139           0.889
L30.termpr_diff             0.038063         0.038242            0.995           0.320
L30.forward1yr_diff        -0.010941         0.016881           -0.648           0.517
L30.ted                     0.001804         0.010930            0.165           0.869
L30.vix                    -0.000113         0.000354           -0.320           0.749
L30.rec_ind                -0.003709         0.013999           -0.265           0.791
L30.1yr-ffr                 0.000882         0.002028            0.435           0.664
L31.yieldsp_diff            0.009064         0.010464            0.866           0.386
L31.termpr_diff             0.032664         0.038244            0.854           0.393
L31.forward1yr_diff        -0.036373         0.016885           -2.154           0.031
L31.ted                     0.004935         0.010922            0.452           0.651
L31.vix                    -0.000178         0.000353           -0.503           0.615
L31.rec_ind                -0.001553         0.014001           -0.111           0.912
L31.1yr-ffr                -0.001616         0.002025           -0.798           0.425
L32.yieldsp_diff            0.001423         0.010465            0.136           0.892
L32.termpr_diff            -0.091296         0.038209           -2.389           0.017
L32.forward1yr_diff         0.040637         0.016873            2.408           0.016
L32.ted                    -0.014878         0.010882           -1.367           0.172
L32.vix                     0.000038         0.000352            0.107           0.915
L32.rec_ind                 0.007645         0.014018            0.545           0.585
L32.1yr-ffr                 0.003735         0.002007            1.861           0.063
L33.yieldsp_diff            0.006314         0.010469            0.603           0.546
L33.termpr_diff             0.031148         0.038225            0.815           0.415
L33.forward1yr_diff        -0.024446         0.016879           -1.448           0.148
L33.ted                     0.009406         0.010890            0.864           0.388
L33.vix                     0.000067         0.000352            0.191           0.849
L33.rec_ind                -0.003611         0.014016           -0.258           0.797
L33.1yr-ffr                -0.002070         0.002006           -1.032           0.302
L34.yieldsp_diff            0.031215         0.010469            2.982           0.003
L34.termpr_diff            -0.106339         0.038203           -2.784           0.005
L34.forward1yr_diff         0.034567         0.016869            2.049           0.040
L34.ted                    -0.003111         0.010897           -0.286           0.775
L34.vix                     0.000108         0.000352            0.306           0.759
L34.rec_ind                -0.006950         0.014011           -0.496           0.620
L34.1yr-ffr                -0.003119         0.002005           -1.556           0.120
L35.yieldsp_diff            0.012124         0.010465            1.159           0.247
L35.termpr_diff            -0.080697         0.038218           -2.111           0.035
L35.forward1yr_diff         0.036750         0.016871            2.178           0.029
L35.ted                    -0.001349         0.010900           -0.124           0.902
L35.vix                    -0.000389         0.000352           -1.103           0.270
L35.rec_ind                 0.006843         0.014003            0.489           0.625
L35.1yr-ffr                 0.002835         0.002005            1.414           0.157
L36.yieldsp_diff            0.014260         0.010464            1.363           0.173
L36.termpr_diff             0.011148         0.038217            0.292           0.771
L36.forward1yr_diff        -0.016356         0.016877           -0.969           0.332
L36.ted                    -0.001240         0.010890           -0.114           0.909
L36.vix                     0.000333         0.000352            0.947           0.344
L36.rec_ind                 0.003969         0.013993            0.284           0.777
L36.1yr-ffr                -0.001540         0.002003           -0.769           0.442
L37.yieldsp_diff            0.009246         0.010463            0.884           0.377
L37.termpr_diff             0.009362         0.038207            0.245           0.806
L37.forward1yr_diff        -0.005773         0.016873           -0.342           0.732
L37.ted                     0.002516         0.010882            0.231           0.817
L37.vix                    -0.000024         0.000352           -0.068           0.946
L37.rec_ind                -0.007666         0.013994           -0.548           0.584
L37.1yr-ffr                 0.001100         0.002004            0.549           0.583
L38.yieldsp_diff            0.008573         0.010438            0.821           0.411
L38.termpr_diff             0.018768         0.038132            0.492           0.623
L38.forward1yr_diff        -0.025010         0.016859           -1.483           0.138
L38.ted                    -0.000870         0.010884           -0.080           0.936
L38.vix                     0.000120         0.000352            0.341           0.733
L38.rec_ind                 0.003810         0.013991            0.272           0.785
L38.1yr-ffr                -0.001973         0.002004           -0.984           0.325
L39.yieldsp_diff            0.012589         0.010438            1.206           0.228
L39.termpr_diff            -0.026152         0.038103           -0.686           0.492
L39.forward1yr_diff         0.007782         0.016849            0.462           0.644
L39.ted                     0.002934         0.010870            0.270           0.787
L39.vix                     0.000061         0.000351            0.173           0.863
L39.rec_ind                -0.002612         0.013987           -0.187           0.852
L39.1yr-ffr                -0.000273         0.002001           -0.137           0.891
L40.yieldsp_diff           -0.018106         0.010410           -1.739           0.082
L40.termpr_diff             0.037018         0.038041            0.973           0.331
L40.forward1yr_diff        -0.007808         0.016822           -0.464           0.643
L40.ted                    -0.001825         0.010865           -0.168           0.867
L40.vix                     0.000012         0.000351            0.033           0.973
L40.rec_ind                 0.000949         0.013987            0.068           0.946
L40.1yr-ffr                 0.005051         0.001999            2.527           0.011
L41.yieldsp_diff           -0.010937         0.010388           -1.053           0.292
L41.termpr_diff            -0.015135         0.037999           -0.398           0.690
L41.forward1yr_diff         0.012330         0.016818            0.733           0.463
L41.ted                     0.024718         0.010859            2.276           0.023
L41.vix                    -0.000352         0.000351           -1.002           0.316
L41.rec_ind                -0.002383         0.013990           -0.170           0.865
L41.1yr-ffr                -0.002274         0.001998           -1.138           0.255
L42.yieldsp_diff           -0.001444         0.010305           -0.140           0.889
L42.termpr_diff             0.033827         0.037751            0.896           0.370
L42.forward1yr_diff        -0.016385         0.016776           -0.977           0.329
L42.ted                    -0.004795         0.010854           -0.442           0.659
L42.vix                     0.000314         0.000351            0.896           0.370
L42.rec_ind                 0.006749         0.013988            0.482           0.629
L42.1yr-ffr                -0.000564         0.001996           -0.282           0.778
L43.yieldsp_diff           -0.009178         0.010148           -0.904           0.366
L43.termpr_diff            -0.019906         0.037328           -0.533           0.594
L43.forward1yr_diff         0.015339         0.016673            0.920           0.358
L43.ted                    -0.011927         0.010847           -1.100           0.272
L43.vix                    -0.000157         0.000348           -0.452           0.651
L43.rec_ind                -0.003152         0.013986           -0.225           0.822
L43.1yr-ffr                -0.000528         0.002001           -0.264           0.792
L44.yieldsp_diff           -0.000170         0.010127           -0.017           0.987
L44.termpr_diff             0.011994         0.037263            0.322           0.748
L44.forward1yr_diff        -0.011699         0.016575           -0.706           0.480
L44.ted                     0.002096         0.010832            0.194           0.847
L44.vix                     0.000369         0.000330            1.119           0.263
L44.rec_ind                 0.003023         0.013984            0.216           0.829
L44.1yr-ffr                 0.000923         0.001997            0.462           0.644
L45.yieldsp_diff            0.002369         0.007631            0.310           0.756
L45.termpr_diff             0.012939         0.031313            0.413           0.679
L45.forward1yr_diff        -0.013310         0.014311           -0.930           0.352
L45.ted                    -0.002961         0.007652           -0.387           0.699
L45.vix                    -0.000370         0.000206           -1.798           0.072
L45.rec_ind                -0.004280         0.009902           -0.432           0.666
L45.1yr-ffr                -0.001258         0.001698           -0.741           0.459
======================================================================================

Results for equation 1yr-ffr
======================================================================================
                         coefficient       std. error           t-stat            prob
--------------------------------------------------------------------------------------
const                       0.005864         0.004440            1.321           0.187
L1.yieldsp_diff             0.250445         0.060305            4.153           0.000
L1.termpr_diff             -0.669987         0.211070           -3.174           0.002
L1.forward1yr_diff          0.373794         0.088663            4.216           0.000
L1.ted                     -0.358616         0.045085           -7.954           0.000
L1.vix                     -0.002175         0.001235           -1.761           0.078
L1.rec_ind                  0.005486         0.058067            0.094           0.925
L1.1yr-ffr                  0.609333         0.010117           60.227           0.000
L2.yieldsp_diff             0.146362         0.060569            2.416           0.016
L2.termpr_diff             -0.122323         0.223285           -0.548           0.584
L2.forward1yr_diff          0.205756         0.098180            2.096           0.036
L2.ted                      0.291444         0.063646            4.579           0.000
L2.vix                      0.003960         0.001983            1.997           0.046
L2.rec_ind                 -0.108018         0.082025           -1.317           0.188
L2.1yr-ffr                  0.083300         0.011839            7.036           0.000
L3.yieldsp_diff             0.140503         0.060759            2.312           0.021
L3.termpr_diff             -0.394740         0.224190           -1.761           0.078
L3.forward1yr_diff          0.280308         0.098745            2.839           0.005
L3.ted                     -0.094587         0.063628           -1.487           0.137
L3.vix                     -0.000929         0.002090           -0.445           0.657
L3.rec_ind                  0.108853         0.082094            1.326           0.185
L3.1yr-ffr                 -0.022601         0.011839           -1.909           0.056
L4.yieldsp_diff            -0.024775         0.061485           -0.403           0.687
L4.termpr_diff             -0.261322         0.224736           -1.163           0.245
L4.forward1yr_diff          0.384566         0.099131            3.879           0.000
L4.ted                      0.275532         0.063633            4.330           0.000
L4.vix                      0.000028         0.002108            0.013           0.989
L4.rec_ind                 -0.128511         0.082120           -1.565           0.118
L4.1yr-ffr                  0.013252         0.011818            1.121           0.262
L5.yieldsp_diff             0.063638         0.061724            1.031           0.303
L5.termpr_diff              0.027966         0.225003            0.124           0.901
L5.forward1yr_diff          0.032793         0.099247            0.330           0.741
L5.ted                     -0.215306         0.063720           -3.379           0.001
L5.vix                     -0.000340         0.002106           -0.161           0.872
L5.rec_ind                  0.022704         0.082168            0.276           0.782
L5.1yr-ffr                 -0.014084         0.011817           -1.192           0.233
L6.yieldsp_diff            -0.163532         0.061966           -2.639           0.008
L6.termpr_diff              0.298584         0.225052            1.327           0.185
L6.forward1yr_diff          0.261804         0.099247            2.638           0.008
L6.ted                      0.122802         0.063760            1.926           0.054
L6.vix                      0.001143         0.002104            0.543           0.587
L6.rec_ind                  0.119269         0.082146            1.452           0.147
L6.1yr-ffr                  0.033897         0.011823            2.867           0.004
L7.yieldsp_diff             0.141908         0.062002            2.289           0.022
L7.termpr_diff             -0.097414         0.225060           -0.433           0.665
L7.forward1yr_diff         -0.037305         0.099267           -0.376           0.707
L7.ted                     -0.136104         0.063711           -2.136           0.033
L7.vix                      0.000532         0.002102            0.253           0.800
L7.rec_ind                 -0.051996         0.082154           -0.633           0.527
L7.1yr-ffr                  0.070642         0.011844            5.964           0.000
L8.yieldsp_diff            -0.003887         0.062094           -0.063           0.950
L8.termpr_diff              0.473571         0.225581            2.099           0.036
L8.forward1yr_diff         -0.076890         0.099420           -0.773           0.439
L8.ted                      0.137305         0.063754            2.154           0.031
L8.vix                     -0.001666         0.002101           -0.793           0.428
L8.rec_ind                 -0.023296         0.082148           -0.284           0.777
L8.1yr-ffr                  0.003071         0.011870            0.259           0.796
L9.yieldsp_diff            -0.117221         0.062030           -1.890           0.059
L9.termpr_diff              0.388197         0.225731            1.720           0.085
L9.forward1yr_diff          0.060283         0.099498            0.606           0.545
L9.ted                      0.009639         0.063827            0.151           0.880
L9.vix                      0.000341         0.002100            0.163           0.871
L9.rec_ind                  0.040395         0.082192            0.491           0.623
L9.1yr-ffr                 -0.042695         0.011863           -3.599           0.000
L10.yieldsp_diff           -0.017370         0.062051           -0.280           0.780
L10.termpr_diff            -0.259832         0.225737           -1.151           0.250
L10.forward1yr_diff         0.287239         0.099532            2.886           0.004
L10.ted                    -0.107599         0.063873           -1.685           0.092
L10.vix                    -0.002490         0.002099           -1.186           0.236
L10.rec_ind                -0.075910         0.082268           -0.923           0.356
L10.1yr-ffr                 0.051176         0.011868            4.312           0.000
L11.yieldsp_diff            0.022583         0.062000            0.364           0.716
L11.termpr_diff             0.262067         0.225643            1.161           0.245
L11.forward1yr_diff        -0.108473         0.099587           -1.089           0.276
L11.ted                    -0.106675         0.063927           -1.669           0.095
L11.vix                     0.003356         0.002098            1.600           0.110
L11.rec_ind                -0.052287         0.082260           -0.636           0.525
L11.1yr-ffr                -0.000239         0.011878           -0.020           0.984
L12.yieldsp_diff            0.139751         0.062017            2.253           0.024
L12.termpr_diff            -0.604657         0.225807           -2.678           0.007
L12.forward1yr_diff         0.242389         0.099624            2.433           0.015
L12.ted                     0.010546         0.063960            0.165           0.869
L12.vix                    -0.000068         0.002097           -0.032           0.974
L12.rec_ind                 0.171408         0.082255            2.084           0.037
L12.1yr-ffr                 0.005668         0.011868            0.478           0.633
L13.yieldsp_diff           -0.058918         0.062005           -0.950           0.342
L13.termpr_diff             0.098945         0.225804            0.438           0.661
L13.forward1yr_diff         0.254963         0.099637            2.559           0.010
L13.ted                     0.152386         0.063983            2.382           0.017
L13.vix                    -0.002321         0.002096           -1.107           0.268
L13.rec_ind                -0.029083         0.082250           -0.354           0.724
L13.1yr-ffr                -0.011999         0.011857           -1.012           0.312
L14.yieldsp_diff           -0.080000         0.061931           -1.292           0.196
L14.termpr_diff             0.234853         0.225586            1.041           0.298
L14.forward1yr_diff        -0.026142         0.099652           -0.262           0.793
L14.ted                     0.108110         0.064029            1.688           0.091
L14.vix                     0.000885         0.002096            0.422           0.673
L14.rec_ind                -0.046795         0.082171           -0.569           0.569
L14.1yr-ffr                 0.160255         0.011857           13.516           0.000
L15.yieldsp_diff            0.081509         0.061958            1.316           0.188
L15.termpr_diff            -0.262123         0.225606           -1.162           0.245
L15.forward1yr_diff         0.103903         0.099666            1.043           0.297
L15.ted                    -0.109127         0.064323           -1.697           0.090
L15.vix                     0.001523         0.002101            0.725           0.469
L15.rec_ind                 0.046393         0.082088            0.565           0.572
L15.1yr-ffr                -0.070566         0.011962           -5.899           0.000
L16.yieldsp_diff            0.112990         0.061932            1.824           0.068
L16.termpr_diff            -0.203299         0.225622           -0.901           0.368
L16.forward1yr_diff         0.118417         0.099653            1.188           0.235
L16.ted                     0.003419         0.064391            0.053           0.958
L16.vix                    -0.003396         0.002105           -1.613           0.107
L16.rec_ind                -0.010583         0.082089           -0.129           0.897
L16.1yr-ffr                -0.024469         0.011981           -2.042           0.041
L17.yieldsp_diff           -0.151092         0.061924           -2.440           0.015
L17.termpr_diff             0.280531         0.225616            1.243           0.214
L17.forward1yr_diff         0.116997         0.099678            1.174           0.240
L17.ted                     0.112265         0.064406            1.743           0.081
L17.vix                     0.000961         0.002105            0.457           0.648
L17.rec_ind                -0.007545         0.082081           -0.092           0.927
L17.1yr-ffr                 0.012884         0.011967            1.077           0.282
L18.yieldsp_diff            0.032634         0.061973            0.527           0.598
L18.termpr_diff            -0.042339         0.225474           -0.188           0.851
L18.forward1yr_diff        -0.059988         0.099646           -0.602           0.547
L18.ted                    -0.136694         0.064396           -2.123           0.034
L18.vix                     0.001741         0.002103            0.828           0.408
L18.rec_ind                 0.067704         0.082179            0.824           0.410
L18.1yr-ffr                -0.009959         0.011902           -0.837           0.403
L19.yieldsp_diff            0.016770         0.061993            0.271           0.787
L19.termpr_diff            -0.043887         0.225416           -0.195           0.846
L19.forward1yr_diff         0.023530         0.099623            0.236           0.813
L19.ted                     0.075670         0.064470            1.174           0.241
L19.vix                    -0.001292         0.002101           -0.615           0.539
L19.rec_ind                -0.038508         0.082259           -0.468           0.640
L19.1yr-ffr                 0.005060         0.011891            0.426           0.670
L20.yieldsp_diff           -0.046991         0.061951           -0.759           0.448
L20.termpr_diff            -0.210906         0.225472           -0.935           0.350
L20.forward1yr_diff         0.236511         0.099605            2.374           0.018
L20.ted                    -0.027575         0.064482           -0.428           0.669
L20.vix                    -0.000929         0.002101           -0.442           0.658
L20.rec_ind                 0.008762         0.082430            0.106           0.915
L20.1yr-ffr                 0.023900         0.011888            2.010           0.044
L21.yieldsp_diff            0.092848         0.061940            1.499           0.134
L21.termpr_diff             0.062522         0.225449            0.277           0.782
L21.forward1yr_diff        -0.065094         0.099625           -0.653           0.514
L21.ted                    -0.118056         0.064510           -1.830           0.067
L21.vix                     0.003056         0.002100            1.455           0.146
L21.rec_ind                -0.174368         0.082418           -2.116           0.034
L21.1yr-ffr                -0.009023         0.011892           -0.759           0.448
L22.yieldsp_diff           -0.197877         0.061985           -3.192           0.001
L22.termpr_diff             0.505554         0.225633            2.241           0.025
L22.forward1yr_diff         0.012306         0.099702            0.123           0.902
L22.ted                     0.280849         0.064535            4.352           0.000
L22.vix                    -0.002889         0.002100           -1.376           0.169
L22.rec_ind                 0.305701         0.082457            3.707           0.000
L22.1yr-ffr                -0.029537         0.011890           -2.484           0.013
L23.yieldsp_diff            0.006707         0.061937            0.108           0.914
L23.termpr_diff            -0.184571         0.225584           -0.818           0.413
L23.forward1yr_diff         0.062522         0.099658            0.627           0.530
L23.ted                    -0.183062         0.064527           -2.837           0.005
L23.vix                     0.002136         0.002099            1.018           0.309
L23.rec_ind                -0.123010         0.082585           -1.490           0.136
L23.1yr-ffr                -0.018423         0.011890           -1.549           0.121
L24.yieldsp_diff           -0.049678         0.061919           -0.802           0.422
L24.termpr_diff             0.050052         0.225425            0.222           0.824
L24.forward1yr_diff         0.158084         0.099593            1.587           0.112
L24.ted                     0.148916         0.064541            2.307           0.021
L24.vix                     0.000456         0.002097            0.218           0.828
L24.rec_ind                -0.023822         0.082687           -0.288           0.773
L24.1yr-ffr                 0.043970         0.011878            3.702           0.000
L25.yieldsp_diff           -0.149131         0.061893           -2.410           0.016
L25.termpr_diff             0.566556         0.225127            2.517           0.012
L25.forward1yr_diff        -0.102516         0.099567           -1.030           0.303
L25.ted                    -0.016894         0.064535           -0.262           0.793
L25.vix                    -0.001563         0.002095           -0.746           0.456
L25.rec_ind                -0.024857         0.082680           -0.301           0.764
L25.1yr-ffr                 0.016704         0.011887            1.405           0.160
L26.yieldsp_diff           -0.270947         0.061917           -4.376           0.000
L26.termpr_diff             0.490741         0.225274            2.178           0.029
L26.forward1yr_diff        -0.058353         0.099546           -0.586           0.558
L26.ted                     0.118746         0.064547            1.840           0.066
L26.vix                     0.001033         0.002094            0.493           0.622
L26.rec_ind                 0.025869         0.082619            0.313           0.754
L26.1yr-ffr                 0.004675         0.011882            0.393           0.694
L27.yieldsp_diff            0.006575         0.061891            0.106           0.915
L27.termpr_diff             0.065104         0.225406            0.289           0.773
L27.forward1yr_diff        -0.068442         0.099487           -0.688           0.491
L27.ted                    -0.255340         0.064494           -3.959           0.000
L27.vix                    -0.000830         0.002093           -0.396           0.692
L27.rec_ind                -0.066908         0.082585           -0.810           0.418
L27.1yr-ffr                -0.016100         0.011879           -1.355           0.175
L28.yieldsp_diff            0.169566         0.061875            2.740           0.006
L28.termpr_diff            -0.506062         0.225429           -2.245           0.025
L28.forward1yr_diff         0.065303         0.099492            0.656           0.512
L28.ted                    -0.032754         0.064476           -0.508           0.611
L28.vix                    -0.001343         0.002092           -0.642           0.521
L28.rec_ind                 0.080117         0.082539            0.971           0.332
L28.1yr-ffr                 0.118475         0.011872            9.979           0.000
L29.yieldsp_diff           -0.033954         0.061875           -0.549           0.583
L29.termpr_diff             0.190851         0.225439            0.847           0.397
L29.forward1yr_diff         0.008831         0.099471            0.089           0.929
L29.ted                    -0.006890         0.064436           -0.107           0.915
L29.vix                     0.002910         0.002091            1.392           0.164
L29.rec_ind                 0.030165         0.082529            0.366           0.715
L29.1yr-ffr                -0.067668         0.011931           -5.672           0.000
L30.yieldsp_diff            0.105662         0.061772            1.711           0.087
L30.termpr_diff            -0.410248         0.225410           -1.820           0.069
L30.forward1yr_diff         0.151311         0.099500            1.521           0.128
L30.ted                     0.035706         0.064422            0.554           0.579
L30.vix                    -0.001544         0.002089           -0.739           0.460
L30.rec_ind                -0.113292         0.082513           -1.373           0.170
L30.1yr-ffr                 0.021292         0.011951            1.782           0.075
L31.yieldsp_diff            0.001795         0.061679            0.029           0.977
L31.termpr_diff             0.006714         0.225423            0.030           0.976
L31.forward1yr_diff         0.034610         0.099523            0.348           0.728
L31.ted                     0.059561         0.064379            0.925           0.355
L31.vix                     0.000308         0.002082            0.148           0.882
L31.rec_ind                 0.196220         0.082524            2.378           0.017
L31.1yr-ffr                 0.030970         0.011937            2.594           0.009
L32.yieldsp_diff           -0.064169         0.061682           -1.040           0.298
L32.termpr_diff             0.133549         0.225214            0.593           0.553
L32.forward1yr_diff         0.077439         0.099454            0.779           0.436
L32.ted                     0.108491         0.064143            1.691           0.091
L32.vix                    -0.000467         0.002077           -0.225           0.822
L32.rec_ind                -0.076565         0.082625           -0.927           0.354
L32.1yr-ffr                 0.019256         0.011833            1.627           0.104
L33.yieldsp_diff            0.099243         0.061709            1.608           0.108
L33.termpr_diff            -0.054208         0.225309           -0.241           0.810
L33.forward1yr_diff        -0.071734         0.099492           -0.721           0.471
L33.ted                    -0.236129         0.064191           -3.679           0.000
L33.vix                     0.001300         0.002076            0.626           0.531
L33.rec_ind                -0.054729         0.082612           -0.662           0.508
L33.1yr-ffr                -0.019030         0.011826           -1.609           0.108
L34.yieldsp_diff           -0.018926         0.061705           -0.307           0.759
L34.termpr_diff             0.096656         0.225178            0.429           0.668
L34.forward1yr_diff         0.053497         0.099432            0.538           0.591
L34.ted                     0.172527         0.064231            2.686           0.007
L34.vix                     0.000091         0.002077            0.044           0.965
L34.rec_ind                -0.021834         0.082582           -0.264           0.791
L34.1yr-ffr                -0.038649         0.011815           -3.271           0.001
L35.yieldsp_diff            0.099779         0.061684            1.618           0.106
L35.termpr_diff            -0.146532         0.225268           -0.650           0.515
L35.forward1yr_diff        -0.120934         0.099441           -1.216           0.224
L35.ted                    -0.132237         0.064249           -2.058           0.040
L35.vix                    -0.003086         0.002076           -1.486           0.137
L35.rec_ind                -0.079828         0.082537           -0.967           0.333
L35.1yr-ffr                 0.004822         0.011820            0.408           0.683
L36.yieldsp_diff            0.023862         0.061679            0.387           0.699
L36.termpr_diff             0.023831         0.225261            0.106           0.916
L36.forward1yr_diff        -0.060414         0.099480           -0.607           0.544
L36.ted                    -0.037106         0.064190           -0.578           0.563
L36.vix                     0.003368         0.002075            1.623           0.105
L36.rec_ind                 0.146132         0.082478            1.772           0.076
L36.1yr-ffr                 0.018086         0.011809            1.532           0.126
L37.yieldsp_diff            0.010928         0.061671            0.177           0.859
L37.termpr_diff            -0.219279         0.225200           -0.974           0.330
L37.forward1yr_diff         0.159477         0.099452            1.604           0.109
L37.ted                     0.159887         0.064144            2.493           0.013
L37.vix                    -0.001075         0.002074           -0.518           0.604
L37.rec_ind                -0.016545         0.082482           -0.201           0.841
L37.1yr-ffr                 0.020244         0.011809            1.714           0.086
L38.yieldsp_diff           -0.084953         0.061523           -1.381           0.167
L38.termpr_diff             0.428095         0.224762            1.905           0.057
L38.forward1yr_diff        -0.169902         0.099372           -1.710           0.087
L38.ted                    -0.081009         0.064154           -1.263           0.207
L38.vix                     0.000125         0.002073            0.061           0.952
L38.rec_ind                -0.003819         0.082466           -0.046           0.963
L38.1yr-ffr                -0.031656         0.011812           -2.680           0.007
L39.yieldsp_diff           -0.037614         0.061525           -0.611           0.541
L39.termpr_diff             0.068593         0.224590            0.305           0.760
L39.forward1yr_diff         0.024061         0.099310            0.242           0.809
L39.ted                     0.003138         0.064072            0.049           0.961
L39.vix                    -0.000998         0.002071           -0.482           0.630
L39.rec_ind                 0.003372         0.082442            0.041           0.967
L39.1yr-ffr                -0.009966         0.011795           -0.845           0.398
L40.yieldsp_diff            0.102395         0.061362            1.669           0.095
L40.termpr_diff            -0.295473         0.224226           -1.318           0.188
L40.forward1yr_diff         0.063456         0.099152            0.640           0.522
L40.ted                    -0.057328         0.064043           -0.895           0.371
L40.vix                    -0.000046         0.002071           -0.022           0.982
L40.rec_ind                 0.067495         0.082440            0.819           0.413
L40.1yr-ffr                -0.006450         0.011780           -0.547           0.584
L41.yieldsp_diff            0.021759         0.061230            0.355           0.722
L41.termpr_diff             0.226154         0.223977            1.010           0.313
L41.forward1yr_diff        -0.003245         0.099131           -0.033           0.974
L41.ted                     0.085752         0.064005            1.340           0.180
L41.vix                     0.002173         0.002069            1.050           0.294
L41.rec_ind                -0.035583         0.082459           -0.432           0.666
L41.1yr-ffr                 0.001006         0.011778            0.085           0.932
L42.yieldsp_diff           -0.069910         0.060743           -1.151           0.250
L42.termpr_diff             0.220305         0.222513            0.990           0.322
L42.forward1yr_diff         0.017192         0.098882            0.174           0.862
L42.ted                    -0.025106         0.063978           -0.392           0.695
L42.vix                    -0.000430         0.002068           -0.208           0.835
L42.rec_ind                 0.006782         0.082448            0.082           0.934
L42.1yr-ffr                 0.076154         0.011767            6.472           0.000
L43.yieldsp_diff           -0.121569         0.059816           -2.032           0.042
L43.termpr_diff            -0.091285         0.220023           -0.415           0.678
L43.forward1yr_diff         0.152623         0.098276            1.553           0.120
L43.ted                     0.155105         0.063937            2.426           0.015
L43.vix                    -0.000825         0.002050           -0.402           0.688
L43.rec_ind                -0.096309         0.082436           -1.168           0.243
L43.1yr-ffr                -0.075640         0.011792           -6.414           0.000
L44.yieldsp_diff           -0.002335         0.059693           -0.039           0.969
L44.termpr_diff             0.378578         0.219640            1.724           0.085
L44.forward1yr_diff        -0.178572         0.097700           -1.828           0.068
L44.ted                    -0.168159         0.063845           -2.634           0.008
L44.vix                     0.001376         0.001946            0.707           0.480
L44.rec_ind                 0.104447         0.082425            1.267           0.205
L44.1yr-ffr                 0.022778         0.011771            1.935           0.053
L45.yieldsp_diff           -0.049625         0.044979           -1.103           0.270
L45.termpr_diff            -0.024398         0.184566           -0.132           0.895
L45.forward1yr_diff         0.110220         0.084354            1.307           0.191
L45.ted                     0.015005         0.045102            0.333           0.739
L45.vix                    -0.002248         0.001214           -1.852           0.064
L45.rec_ind                -0.050319         0.058367           -0.862           0.389
L45.1yr-ffr                 0.018344         0.010010            1.833           0.067
======================================================================================

Correlation matrix of residuals
                   yieldsp_diff  termpr_diff  forward1yr_diff       ted       vix   rec_ind   1yr-ffr
yieldsp_diff           1.000000     0.775606         0.496012  0.392611 -0.071889 -0.009031  0.090175
termpr_diff            0.775606     1.000000         0.859908 -0.040809 -0.130975 -0.017659  0.152171
forward1yr_diff        0.496012     0.859908         1.000000 -0.263808 -0.144755 -0.020393  0.206406
ted                    0.392611    -0.040809        -0.263808  1.000000  0.074689  0.012988 -0.109363
vix                   -0.071889    -0.130975        -0.144755  0.074689  1.000000  0.010336 -0.027277
rec_ind               -0.009031    -0.017659        -0.020393  0.012988  0.010336  1.000000 -0.024552
1yr-ffr                0.090175     0.152171         0.206406 -0.109363 -0.027277 -0.024552  1.000000

Durbin-Watson Statistic

The author claims that the durbin-watson statistic for all the variables in the VAR system is 2, implying that they are serially uncorrelated.

In [ ]:
from statsmodels.stats.stattools import durbin_watson
out = durbin_watson(res.resid)

for col, val in zip(stat_train.columns, out):
    print((col), ':', round(val, 2))
yieldsp_diff : 2.0
termpr_diff : 2.0
forward1yr_diff : 2.0
ted : 2.0
vix : 2.0
rec_ind : 2.0
1yr-ffr : 2.0

Forecasting using the VAR model

In [ ]:
lag_order = res.k_ar

train, test = np.split(stat1, [int(.95 *len(stat1))])

stat_test = test[['yieldsp_diff','termpr_diff', 'forward1yr_diff', 'ted', 'vix', 'rec_ind', '1yr-ffr']]

input_data = stat_test.values[-lag_order:]

pred = res.forecast(y=input_data, steps = len(test))
pred1 = pd.DataFrame(pred, index=stat_test.index, columns= stat_test.columns + '_pred')
pred1.head()
Out[ ]:
yieldsp_diff_pred termpr_diff_pred forward1yr_diff_pred ted_pred vix_pred rec_ind_pred 1yr-ffr_pred
Date
2019-03-20 0.018153 0.004315 0.006912 0.141936 26.880096 0.993790 0.079835
2019-03-21 0.001601 0.001717 -0.001114 0.145909 26.685031 0.985281 0.089191
2019-03-22 0.001071 -0.003120 -0.002581 0.143991 26.758351 0.974677 0.083892
2019-03-23 0.017312 0.002926 0.001307 0.155681 26.688817 0.971607 0.083033
2019-03-24 0.016738 0.003896 0.003525 0.163486 26.374960 0.963143 0.110345
In [ ]:
merge0 = pd.merge(test['yieldsp'], pred1['yieldsp_diff_pred'].to_frame(), how='inner', 
                  left_index=True, right_index=True)
merge0.head()
Out[ ]:
yieldsp yieldsp_diff_pred
Date
2019-03-20 0.06 0.018153
2019-03-21 0.05 0.001601
2019-03-22 -0.02 0.001071
2019-03-23 -0.03 0.017312
2019-03-24 -0.03 0.016738
In [ ]:
def invert_diff(history, yhat, interval=1):
    return yhat + history[-interval]

invert0 = invert_diff(merge0['yieldsp'], merge0['yieldsp_diff_pred'], 1).to_frame()
invert0.rename(columns={'yieldsp_diff_pred':'invert_yieldsp_var'}, inplace=True)
merge_invert0 = pd.merge(merge0, invert0, how='inner', left_index=True, 
                         right_index=True).drop(['yieldsp_diff_pred'], axis = 1)
merge_invert0.tail()
Out[ ]:
yieldsp invert_yieldsp_var
Date
2020-09-27 0.56 0.589442
2020-09-28 0.56 0.589444
2020-09-29 0.57 0.589446
2020-09-30 0.59 0.589448
2020-10-01 0.59 0.589450
In [ ]:
error = mean_squared_error(merge_invert0['yieldsp'], merge_invert0['invert_yieldsp_var'])
rmse = sqrt(error)
print('RMSE: %.5f' % rmse)
RMSE: 0.50267

IRF and FEVD

The author has examined the properties of VAR by doing structural analyses using impulse responses and forecast error variance decomposition.

Impulse Response Function (IRF)

IRFs trace out the time path of the effects of an exogenous shock $\epsilon_t$ to one (or more) of the endogenous variables on some or all of the other variables in a VAR system given that no future innovations are present in the system.

In [ ]:
irf1 = res.irf(10)
irf1.plot(response ='yieldsp_diff');

The impulse response of shocks on $\Delta yieldsp$ decays to 0 by the tenth period, signifying that one-time innovations don’t have long-term ramifications on the paths of $\Delta yieldsp$.

Forecast Error Variance Decomposition (FEVD)

Another way to characterize the dynamics associated with the VAR is by computing the FEVD from the VAR model.

In [ ]:
fevd = res.fevd(5)
fevd.summary()
FEVD for yieldsp_diff
     yieldsp_diff  termpr_diff  forward1yr_diff       ted       vix   rec_ind   1yr-ffr
0        1.000000     0.000000         0.000000  0.000000  0.000000  0.000000  0.000000
1        0.998289     0.000920         0.000108  0.000105  0.000460  0.000118  0.000001
2        0.996227     0.001186         0.000123  0.001400  0.000642  0.000341  0.000082
3        0.992569     0.002648         0.000872  0.001410  0.001018  0.001257  0.000225
4        0.988423     0.002771         0.003203  0.001509  0.002160  0.001520  0.000414

FEVD for termpr_diff
     yieldsp_diff  termpr_diff  forward1yr_diff       ted       vix   rec_ind   1yr-ffr
0        0.601565     0.398435         0.000000  0.000000  0.000000  0.000000  0.000000
1        0.612890     0.386126         0.000767  0.000000  0.000044  0.000171  0.000002
2        0.613462     0.384468         0.000915  0.000329  0.000417  0.000191  0.000218
3        0.613674     0.382892         0.000925  0.000623  0.000595  0.001073  0.000218
4        0.612469     0.382070         0.002274  0.000671  0.001014  0.001256  0.000246

FEVD for forward1yr_diff
     yieldsp_diff  termpr_diff  forward1yr_diff       ted       vix   rec_ind   1yr-ffr
0        0.246028     0.566751         0.187221  0.000000  0.000000  0.000000  0.000000
1        0.237748     0.559402         0.202323  0.000171  0.000038  0.000295  0.000023
2        0.237248     0.556892         0.204534  0.000208  0.000389  0.000485  0.000244
3        0.237420     0.555315         0.205698  0.000270  0.000501  0.000546  0.000249
4        0.237143     0.554890         0.206130  0.000302  0.000535  0.000740  0.000260

FEVD for ted
     yieldsp_diff  termpr_diff  forward1yr_diff       ted       vix   rec_ind   1yr-ffr
0        0.154143     0.299286         0.011648  0.534923  0.000000  0.000000  0.000000
1        0.207522     0.261438         0.005637  0.523909  0.001386  0.000003  0.000106
2        0.222410     0.246994         0.003841  0.524295  0.002085  0.000255  0.000120
3        0.229285     0.239252         0.002993  0.524909  0.003182  0.000257  0.000122
4        0.232609     0.228853         0.002487  0.530475  0.005186  0.000244  0.000146

FEVD for vix
     yieldsp_diff  termpr_diff  forward1yr_diff       ted       vix   rec_ind   1yr-ffr
0        0.005168     0.014200         0.002008  0.002022  0.976602  0.000000  0.000000
1        0.004563     0.014656         0.002405  0.002655  0.975633  0.000015  0.000074
2        0.005347     0.015347         0.003274  0.004585  0.971030  0.000273  0.000146
3        0.006059     0.015577         0.004088  0.006628  0.966286  0.001164  0.000198
4        0.007488     0.016030         0.004500  0.007693  0.962036  0.001844  0.000409

FEVD for rec_ind
     yieldsp_diff  termpr_diff  forward1yr_diff       ted       vix   rec_ind   1yr-ffr
0        0.000082     0.000285         0.000055  0.000079  0.000049  0.999450  0.000000
1        0.000042     0.000316         0.000041  0.000042  0.000119  0.999407  0.000032
2        0.000044     0.000228         0.000038  0.000040  0.000240  0.999160  0.000250
3        0.000054     0.000174         0.000031  0.000034  0.000285  0.998098  0.001325
4        0.000163     0.000141         0.000039  0.000134  0.000360  0.997239  0.001924

FEVD for 1yr-ffr
     yieldsp_diff  termpr_diff  forward1yr_diff       ted       vix   rec_ind   1yr-ffr
0        0.008131     0.016971         0.021608  0.006209  0.000024  0.000392  0.946664
1        0.009877     0.027698         0.029947  0.015842  0.000162  0.000364  0.916111
2        0.013272     0.034839         0.035847  0.019176  0.000146  0.000657  0.896063
3        0.015751     0.042452         0.043225  0.023216  0.000197  0.000767  0.874392
4        0.017084     0.048909         0.051950  0.022793  0.000283  0.001233  0.857749


Multilayer Perceptron (MLP)

The author has made various MLP network by varying the number of neurons (1,3 and 5) with the lag period = 3. The author has also run every experiment 8 times.

So, i have set up a generic function to model all the MLPs.

Can defintely improve here; do for a bunch of lag periods and neurons, this is a very shallow network, can improve significantly using deeper networks.

In [ ]:
def parser(x):
    return datetime.strptime('190'+x, '%Y-%m')

def timeseries_to_supervised(data, lag=1):
    df = DataFrame(data)
    columns = [df.shift(i) for i in range(1, lag+1)]
    columns.append(df)
    df = pd.concat(columns, axis=1)
    df = df.drop(0)
    return df

def difference(dataset, interval=1):
    diff = list()
    for i in range(interval, len(dataset)):
        value = dataset[i] - dataset[i - interval]
        diff.append(value)
    return Series(diff)

def inverse_difference(history, yhat, interval=1):
    return yhat + history[-interval]

def scale(train, test):
    # fit scaler
    scaler = MinMaxScaler(feature_range=(-1, 1))
    scaler = scaler.fit(train)
    
    # transform train
    train = train.reshape(train.shape[0], train.shape[1])
    train_scaled = scaler.transform(train)
    
    # transform test
    test = test.reshape(test.shape[0], test.shape[1])
    test_scaled = scaler.transform(test)
    return scaler, train_scaled, test_scaled

def invert_scale(scaler, X, yhat):
    new_row = [x for x in X] + [yhat]
    array = np.array(new_row)
    array = array.reshape(1, len(array))
    inverted = scaler.inverse_transform(array)
    return inverted[0, -1]

def evaluate(model, raw_data, scaled_dataset, scaler, offset, batch_size):
    
    X, y = scaled_dataset[:,0:-1], scaled_dataset[:,-1]
    output = model.predict(X, batch_size=batch_size)
    
    predictions = list()
    for i in range(len(output)):
        yhat = output[i,0]
        yhat = invert_scale(scaler, X[i], yhat)
        yhat = yhat + raw_data[i]
        
        predictions.append(yhat)
        
    #evaluation metric
    rmse = sqrt(mean_squared_error(raw_data[1:], predictions))
    return rmse

# fit an MLP network based on the parameters
def fit(train, test, raw, scaler, batch_size, nb_epoch, neurons):
    X, y = train[:, 0:-1], train[:, -1]
    
    #building the model architecture
    model = Sequential()
    model.add(Dense(neurons, activation='relu', input_dim=X.shape[1]))
    model.add(Dense(1))
    model.compile(loss='mean_squared_error', optimizer='adam')
    
    # fit model
    train_rmse, test_rmse = list(), list()
    for i in range(nb_epoch):
        model.fit(X, y, epochs=1, batch_size=batch_size, verbose=0, shuffle=False)
        
        raw_train = raw[-(len(train)+len(test)+1):-len(test)]
        train_rmse.append(evaluate(model, raw_train, train, scaler, 0, batch_size))
        
        raw_test = raw[-(len(test)+1):]
        test_rmse.append(evaluate(model, raw_test, test, scaler, 0, batch_size))
    history = DataFrame()
    history['train'], history['test'] = train_rmse, test_rmse
    return history

def fit_model(train, batch_size, nb_epoch, neurons):
    X, y = train[:, 0:-1], train[:, -1]
    model = Sequential()
    model.add(Dense(neurons, activation='relu', input_dim=X.shape[1]))
    model.add(Dense(1))
    model.compile(loss='mean_squared_error', optimizer='adam')
    model.fit(X, y, epochs=nb_epoch, batch_size=batch_size, verbose=1, shuffle=False)
    return model

Building, training and evaluating the various MLP network

In [ ]:
# building, training and evaluating the various MLP networks

def experiment(repeats, series, epochs, lag, neurons):
    
    raw_values = series.values
    diff_values = difference(raw_values, 1)
    
    supervised = timeseries_to_supervised(diff_values, lag)
    supervised_values = supervised.values[lag:,:]
    
    train, test = supervised_values[0:-365], supervised_values[-365:]
    scaler, train_scaled, test_scaled = scale(train, test)
    
    error_scores = list()
    for r in range(repeats):
        
        batch_size = 16
        model = fit_model(train_scaled, batch_size, epochs, neurons)
        
        test_reshaped = test_scaled[:,0:-1]
        output = model.predict(test_reshaped, batch_size=batch_size)
        predictions = list()
        for i in range(len(output)):
            yhat = output[i,0]
            X = test_scaled[i, 0:-1]
            yhat = invert_scale(scaler, X, yhat)
            yhat = inverse_difference(raw_values, yhat, len(test_scaled)+1-i)
            
            predictions.append(yhat)
            
        rmse = sqrt(mean_squared_error(raw_values[-365:], predictions))
        
        print('%d) Test RMSE: %.3f' % (r+1, rmse))
        error_scores.append(rmse)
    return error_scores

series = yieldsp

#config variables for the various setups 0
repeats = 8
results = DataFrame()
epochs = 10
neurons = [1, 3, 5]

for n in neurons:
    results[str(n)] = experiment(repeats, series, epochs, n, n)
    
print(results.describe())

results.boxplot()
pyplot.savefig('boxplot_neurons_lag.png')
Epoch 1/10
586/586 [==============================] - 1s 821us/step - loss: 0.0400
Epoch 2/10
586/586 [==============================] - 1s 975us/step - loss: 0.0121
Epoch 3/10
586/586 [==============================] - 0s 838us/step - loss: 0.0121
Epoch 4/10
586/586 [==============================] - 0s 835us/step - loss: 0.0121
Epoch 5/10
586/586 [==============================] - 0s 789us/step - loss: 0.0120
Epoch 6/10
586/586 [==============================] - 0s 801us/step - loss: 0.0119
Epoch 7/10
586/586 [==============================] - 0s 819us/step - loss: 0.0119
Epoch 8/10
586/586 [==============================] - 0s 800us/step - loss: 0.0119
Epoch 9/10
586/586 [==============================] - 0s 837us/step - loss: 0.0119
Epoch 10/10
586/586 [==============================] - 0s 790us/step - loss: 0.0119
1) Test RMSE: 0.056
Epoch 1/10
586/586 [==============================] - 1s 790us/step - loss: 0.1843
Epoch 2/10
586/586 [==============================] - 0s 804us/step - loss: 0.0161
Epoch 3/10
586/586 [==============================] - 0s 817us/step - loss: 0.0141
Epoch 4/10
586/586 [==============================] - 0s 806us/step - loss: 0.0132
Epoch 5/10
586/586 [==============================] - 0s 833us/step - loss: 0.0127
Epoch 6/10
586/586 [==============================] - 0s 825us/step - loss: 0.0124
Epoch 7/10
586/586 [==============================] - 0s 822us/step - loss: 0.0122
Epoch 8/10
586/586 [==============================] - 0s 810us/step - loss: 0.0121
Epoch 9/10
586/586 [==============================] - 0s 808us/step - loss: 0.0120
Epoch 10/10
586/586 [==============================] - 1s 884us/step - loss: 0.0120
2) Test RMSE: 0.056
Epoch 1/10
586/586 [==============================] - 1s 803us/step - loss: 0.0529
Epoch 2/10
586/586 [==============================] - 0s 792us/step - loss: 0.0303
Epoch 3/10
586/586 [==============================] - 0s 834us/step - loss: 0.0237
Epoch 4/10
586/586 [==============================] - 0s 815us/step - loss: 0.0191
Epoch 5/10
586/586 [==============================] - 0s 848us/step - loss: 0.0162
Epoch 6/10
586/586 [==============================] - 0s 804us/step - loss: 0.0144
Epoch 7/10
586/586 [==============================] - 0s 810us/step - loss: 0.0133
Epoch 8/10
586/586 [==============================] - 0s 820us/step - loss: 0.0127
Epoch 9/10
586/586 [==============================] - 0s 780us/step - loss: 0.0124
Epoch 10/10
586/586 [==============================] - 0s 825us/step - loss: 0.0122
3) Test RMSE: 0.056
Epoch 1/10
586/586 [==============================] - 1s 803us/step - loss: 0.0244
Epoch 2/10
586/586 [==============================] - 0s 797us/step - loss: 0.0124
Epoch 3/10
586/586 [==============================] - 0s 817us/step - loss: 0.0122
Epoch 4/10
586/586 [==============================] - 0s 836us/step - loss: 0.0121
Epoch 5/10
586/586 [==============================] - 0s 796us/step - loss: 0.0121
Epoch 6/10
586/586 [==============================] - 0s 805us/step - loss: 0.0120
Epoch 7/10
586/586 [==============================] - 0s 839us/step - loss: 0.0120
Epoch 8/10
586/586 [==============================] - 0s 828us/step - loss: 0.0120
Epoch 9/10
586/586 [==============================] - 0s 833us/step - loss: 0.0120
Epoch 10/10
586/586 [==============================] - 0s 813us/step - loss: 0.0120
4) Test RMSE: 0.056
Epoch 1/10
586/586 [==============================] - 1s 791us/step - loss: 0.0388
Epoch 2/10
586/586 [==============================] - 1s 854us/step - loss: 0.0118
Epoch 3/10
586/586 [==============================] - 0s 811us/step - loss: 0.0118
Epoch 4/10
586/586 [==============================] - 0s 798us/step - loss: 0.0118
Epoch 5/10
586/586 [==============================] - 0s 809us/step - loss: 0.0118
Epoch 6/10
586/586 [==============================] - 0s 814us/step - loss: 0.0118
Epoch 7/10
586/586 [==============================] - 0s 825us/step - loss: 0.0118
Epoch 8/10
586/586 [==============================] - 0s 820us/step - loss: 0.0118
Epoch 9/10
586/586 [==============================] - 0s 810us/step - loss: 0.0118
Epoch 10/10
586/586 [==============================] - 0s 819us/step - loss: 0.0118
5) Test RMSE: 0.056
Epoch 1/10
586/586 [==============================] - 1s 765us/step - loss: 0.0359
Epoch 2/10
586/586 [==============================] - 0s 805us/step - loss: 0.0120
Epoch 3/10
586/586 [==============================] - 0s 794us/step - loss: 0.0120
Epoch 4/10
586/586 [==============================] - 0s 813us/step - loss: 0.0120
Epoch 5/10
586/586 [==============================] - 0s 814us/step - loss: 0.0120
Epoch 6/10
586/586 [==============================] - 0s 836us/step - loss: 0.0120
Epoch 7/10
586/586 [==============================] - 0s 812us/step - loss: 0.0120
Epoch 8/10
586/586 [==============================] - 1s 873us/step - loss: 0.0120
Epoch 9/10
586/586 [==============================] - 0s 798us/step - loss: 0.0120
Epoch 10/10
586/586 [==============================] - 0s 817us/step - loss: 0.0120
6) Test RMSE: 0.056
Epoch 1/10
586/586 [==============================] - 1s 789us/step - loss: 0.0183
Epoch 2/10
586/586 [==============================] - 0s 828us/step - loss: 0.0134
Epoch 3/10
586/586 [==============================] - 0s 807us/step - loss: 0.0127
Epoch 4/10
586/586 [==============================] - 0s 790us/step - loss: 0.0123
Epoch 5/10
586/586 [==============================] - 0s 807us/step - loss: 0.0121
Epoch 6/10
586/586 [==============================] - 0s 844us/step - loss: 0.0120
Epoch 7/10
586/586 [==============================] - 0s 825us/step - loss: 0.0120
Epoch 8/10
586/586 [==============================] - 0s 825us/step - loss: 0.0120
Epoch 9/10
586/586 [==============================] - 0s 799us/step - loss: 0.0120
Epoch 10/10
586/586 [==============================] - 0s 834us/step - loss: 0.0119
7) Test RMSE: 0.056
Epoch 1/10
586/586 [==============================] - 1s 817us/step - loss: 0.0460
Epoch 2/10
586/586 [==============================] - 0s 802us/step - loss: 0.0155
Epoch 3/10
586/586 [==============================] - 0s 820us/step - loss: 0.0150
Epoch 4/10
586/586 [==============================] - 0s 818us/step - loss: 0.0146
Epoch 5/10
586/586 [==============================] - 1s 2ms/step - loss: 0.0142
Epoch 6/10
586/586 [==============================] - 1s 1ms/step - loss: 0.0139
Epoch 7/10
586/586 [==============================] - 1s 984us/step - loss: 0.0135
Epoch 8/10
586/586 [==============================] - 0s 843us/step - loss: 0.0132
Epoch 9/10
586/586 [==============================] - 1s 1ms/step - loss: 0.0129
Epoch 10/10
586/586 [==============================] - 1s 850us/step - loss: 0.0127
8) Test RMSE: 0.056
Epoch 1/10
586/586 [==============================] - 1s 984us/step - loss: 0.0197
Epoch 2/10
586/586 [==============================] - 0s 798us/step - loss: 0.0160
Epoch 3/10
586/586 [==============================] - 0s 820us/step - loss: 0.0141
Epoch 4/10
586/586 [==============================] - 0s 818us/step - loss: 0.0133
Epoch 5/10
586/586 [==============================] - 0s 846us/step - loss: 0.0129
Epoch 6/10
586/586 [==============================] - 1s 1ms/step - loss: 0.0126
Epoch 7/10
586/586 [==============================] - 0s 816us/step - loss: 0.0125
Epoch 8/10
586/586 [==============================] - 0s 824us/step - loss: 0.0124
Epoch 9/10
586/586 [==============================] - 0s 842us/step - loss: 0.0123
Epoch 10/10
586/586 [==============================] - 0s 833us/step - loss: 0.0122
1) Test RMSE: 0.055
Epoch 1/10
586/586 [==============================] - 1s 773us/step - loss: 0.0273
Epoch 2/10
586/586 [==============================] - 0s 812us/step - loss: 0.0154
Epoch 3/10
586/586 [==============================] - 1s 999us/step - loss: 0.0136
Epoch 4/10
586/586 [==============================] - 0s 841us/step - loss: 0.0128
Epoch 5/10
586/586 [==============================] - 0s 832us/step - loss: 0.0125
Epoch 6/10
586/586 [==============================] - 1s 874us/step - loss: 0.0123
Epoch 7/10
586/586 [==============================] - 0s 840us/step - loss: 0.0122
Epoch 8/10
586/586 [==============================] - 0s 848us/step - loss: 0.0121
Epoch 9/10
586/586 [==============================] - 1s 877us/step - loss: 0.0121
Epoch 10/10
586/586 [==============================] - 1s 893us/step - loss: 0.0120
2) Test RMSE: 0.055
Epoch 1/10
586/586 [==============================] - 1s 813us/step - loss: 0.0191
Epoch 2/10
586/586 [==============================] - 0s 818us/step - loss: 0.0137
Epoch 3/10
586/586 [==============================] - 1s 1ms/step - loss: 0.0123
Epoch 4/10
586/586 [==============================] - 0s 817us/step - loss: 0.0121
Epoch 5/10
586/586 [==============================] - 1s 859us/step - loss: 0.0121
Epoch 6/10
586/586 [==============================] - 0s 836us/step - loss: 0.0121
Epoch 7/10
586/586 [==============================] - 1s 859us/step - loss: 0.0120
Epoch 8/10
586/586 [==============================] - 0s 821us/step - loss: 0.0120
Epoch 9/10
586/586 [==============================] - 1s 879us/step - loss: 0.0120
Epoch 10/10
586/586 [==============================] - 0s 799us/step - loss: 0.0120
3) Test RMSE: 0.055
Epoch 1/10
586/586 [==============================] - 1s 771us/step - loss: 0.0180
Epoch 2/10
586/586 [==============================] - 0s 825us/step - loss: 0.0133
Epoch 3/10
586/586 [==============================] - 0s 823us/step - loss: 0.0130
Epoch 4/10
586/586 [==============================] - 1s 860us/step - loss: 0.0128
Epoch 5/10
586/586 [==============================] - 0s 829us/step - loss: 0.0127
Epoch 6/10
586/586 [==============================] - 0s 838us/step - loss: 0.0126
Epoch 7/10
586/586 [==============================] - 0s 833us/step - loss: 0.0125
Epoch 8/10
586/586 [==============================] - 1s 1ms/step - loss: 0.0125
Epoch 9/10
586/586 [==============================] - 1s 886us/step - loss: 0.0124
Epoch 10/10
586/586 [==============================] - 0s 845us/step - loss: 0.0124
4) Test RMSE: 0.056
Epoch 1/10
586/586 [==============================] - 1s 815us/step - loss: 0.1714
Epoch 2/10
586/586 [==============================] - 0s 763us/step - loss: 0.0174
Epoch 3/10
586/586 [==============================] - 0s 819us/step - loss: 0.0153
Epoch 4/10
586/586 [==============================] - 0s 832us/step - loss: 0.0145
Epoch 5/10
586/586 [==============================] - 0s 839us/step - loss: 0.0140
Epoch 6/10
586/586 [==============================] - 0s 836us/step - loss: 0.0136
Epoch 7/10
586/586 [==============================] - 0s 819us/step - loss: 0.0133
Epoch 8/10
586/586 [==============================] - 1s 879us/step - loss: 0.0130
Epoch 9/10
586/586 [==============================] - 1s 855us/step - loss: 0.0128
Epoch 10/10
586/586 [==============================] - 1s 849us/step - loss: 0.0127
5) Test RMSE: 0.056
Epoch 1/10
586/586 [==============================] - 1s 811us/step - loss: 0.0426
Epoch 2/10
586/586 [==============================] - 0s 802us/step - loss: 0.0127
Epoch 3/10
586/586 [==============================] - 0s 841us/step - loss: 0.0126
Epoch 4/10
586/586 [==============================] - 0s 835us/step - loss: 0.0126
Epoch 5/10
586/586 [==============================] - 1s 850us/step - loss: 0.0125
Epoch 6/10
586/586 [==============================] - 0s 838us/step - loss: 0.0125
Epoch 7/10
586/586 [==============================] - 0s 831us/step - loss: 0.0124
Epoch 8/10
586/586 [==============================] - 1s 855us/step - loss: 0.0124
Epoch 9/10
586/586 [==============================] - 0s 830us/step - loss: 0.0123
Epoch 10/10
586/586 [==============================] - 0s 836us/step - loss: 0.0123
6) Test RMSE: 0.056
Epoch 1/10
586/586 [==============================] - 1s 797us/step - loss: 0.0399
Epoch 2/10
586/586 [==============================] - 0s 820us/step - loss: 0.0126
Epoch 3/10
586/586 [==============================] - 0s 814us/step - loss: 0.0123
Epoch 4/10
586/586 [==============================] - 0s 829us/step - loss: 0.0122
Epoch 5/10
586/586 [==============================] - 0s 842us/step - loss: 0.0122
Epoch 6/10
586/586 [==============================] - 1s 871us/step - loss: 0.0121
Epoch 7/10
586/586 [==============================] - 0s 844us/step - loss: 0.0121
Epoch 8/10
586/586 [==============================] - 1s 851us/step - loss: 0.0121
Epoch 9/10
586/586 [==============================] - 0s 825us/step - loss: 0.0121
Epoch 10/10
586/586 [==============================] - 1s 876us/step - loss: 0.0121
7) Test RMSE: 0.056
Epoch 1/10
586/586 [==============================] - 1s 780us/step - loss: 0.0223
Epoch 2/10
586/586 [==============================] - 0s 795us/step - loss: 0.0132
Epoch 3/10
586/586 [==============================] - 0s 817us/step - loss: 0.0127
Epoch 4/10
586/586 [==============================] - 0s 824us/step - loss: 0.0125
Epoch 5/10
586/586 [==============================] - 0s 843us/step - loss: 0.0124
Epoch 6/10
586/586 [==============================] - 0s 834us/step - loss: 0.0123
Epoch 7/10
586/586 [==============================] - 0s 831us/step - loss: 0.0122
Epoch 8/10
586/586 [==============================] - 1s 874us/step - loss: 0.0122
Epoch 9/10
586/586 [==============================] - 0s 849us/step - loss: 0.0121
Epoch 10/10
586/586 [==============================] - 0s 841us/step - loss: 0.0121
8) Test RMSE: 0.056
Epoch 1/10
586/586 [==============================] - 1s 818us/step - loss: 0.0438
Epoch 2/10
586/586 [==============================] - 0s 812us/step - loss: 0.0127
Epoch 3/10
586/586 [==============================] - 0s 822us/step - loss: 0.0124
Epoch 4/10
586/586 [==============================] - 1s 875us/step - loss: 0.0122
Epoch 5/10
586/586 [==============================] - 0s 850us/step - loss: 0.0120
Epoch 6/10
586/586 [==============================] - 1s 863us/step - loss: 0.0119
Epoch 7/10
586/586 [==============================] - 0s 848us/step - loss: 0.0118
Epoch 8/10
586/586 [==============================] - 0s 844us/step - loss: 0.0117
Epoch 9/10
586/586 [==============================] - 0s 837us/step - loss: 0.0117
Epoch 10/10
586/586 [==============================] - 0s 847us/step - loss: 0.0116
1) Test RMSE: 0.056
Epoch 1/10
586/586 [==============================] - 1s 812us/step - loss: 0.0249
Epoch 2/10
586/586 [==============================] - 0s 836us/step - loss: 0.0157
Epoch 3/10
586/586 [==============================] - 0s 801us/step - loss: 0.0135
Epoch 4/10
586/586 [==============================] - 0s 836us/step - loss: 0.0126
Epoch 5/10
586/586 [==============================] - 1s 872us/step - loss: 0.0122
Epoch 6/10
586/586 [==============================] - 1s 853us/step - loss: 0.0120
Epoch 7/10
586/586 [==============================] - 1s 875us/step - loss: 0.0119
Epoch 8/10
586/586 [==============================] - 1s 858us/step - loss: 0.0118
Epoch 9/10
586/586 [==============================] - 1s 858us/step - loss: 0.0118
Epoch 10/10
586/586 [==============================] - 1s 845us/step - loss: 0.0118
2) Test RMSE: 0.055
Epoch 1/10
586/586 [==============================] - 1s 817us/step - loss: 0.0523
Epoch 2/10
586/586 [==============================] - 0s 793us/step - loss: 0.0174
Epoch 3/10
586/586 [==============================] - 0s 823us/step - loss: 0.0146
Epoch 4/10
586/586 [==============================] - 0s 838us/step - loss: 0.0138
Epoch 5/10
586/586 [==============================] - 0s 847us/step - loss: 0.0134
Epoch 6/10
586/586 [==============================] - 0s 835us/step - loss: 0.0132
Epoch 7/10
586/586 [==============================] - 0s 829us/step - loss: 0.0130
Epoch 8/10
586/586 [==============================] - 0s 829us/step - loss: 0.0129
Epoch 9/10
586/586 [==============================] - 0s 849us/step - loss: 0.0127
Epoch 10/10
586/586 [==============================] - 1s 868us/step - loss: 0.0126
3) Test RMSE: 0.055
Epoch 1/10
586/586 [==============================] - 1s 812us/step - loss: 0.0233
Epoch 2/10
586/586 [==============================] - 0s 810us/step - loss: 0.0134
Epoch 3/10
586/586 [==============================] - 0s 848us/step - loss: 0.0131
Epoch 4/10
586/586 [==============================] - 0s 834us/step - loss: 0.0129
Epoch 5/10
586/586 [==============================] - 1s 876us/step - loss: 0.0127
Epoch 6/10
586/586 [==============================] - 1s 883us/step - loss: 0.0126
Epoch 7/10
586/586 [==============================] - 1s 2ms/step - loss: 0.0125
Epoch 8/10
586/586 [==============================] - 1s 1ms/step - loss: 0.0124
Epoch 9/10
586/586 [==============================] - 1s 1ms/step - loss: 0.0123
Epoch 10/10
586/586 [==============================] - 0s 840us/step - loss: 0.0123
4) Test RMSE: 0.055
Epoch 1/10
586/586 [==============================] - 1s 819us/step - loss: 0.0171
Epoch 2/10
586/586 [==============================] - 1s 995us/step - loss: 0.0141
Epoch 3/10
586/586 [==============================] - 1s 857us/step - loss: 0.0132
Epoch 4/10
586/586 [==============================] - 1s 858us/step - loss: 0.0127
Epoch 5/10
586/586 [==============================] - 0s 824us/step - loss: 0.0125
Epoch 6/10
586/586 [==============================] - 1s 851us/step - loss: 0.0123
Epoch 7/10
586/586 [==============================] - 1s 940us/step - loss: 0.0122
Epoch 8/10
586/586 [==============================] - 1s 883us/step - loss: 0.0122
Epoch 9/10
586/586 [==============================] - 1s 883us/step - loss: 0.0121
Epoch 10/10
586/586 [==============================] - 1s 893us/step - loss: 0.0121
5) Test RMSE: 0.055
Epoch 1/10
586/586 [==============================] - 1s 819us/step - loss: 0.0807
Epoch 2/10
586/586 [==============================] - 1s 982us/step - loss: 0.0150
Epoch 3/10
586/586 [==============================] - 1s 887us/step - loss: 0.0139
Epoch 4/10
586/586 [==============================] - 1s 885us/step - loss: 0.0133
Epoch 5/10
586/586 [==============================] - 1s 876us/step - loss: 0.0129
Epoch 6/10
586/586 [==============================] - 0s 847us/step - loss: 0.0126
Epoch 7/10
586/586 [==============================] - 1s 860us/step - loss: 0.0123
Epoch 8/10
586/586 [==============================] - 1s 874us/step - loss: 0.0121
Epoch 9/10
586/586 [==============================] - 1s 893us/step - loss: 0.0120
Epoch 10/10
586/586 [==============================] - 0s 826us/step - loss: 0.0119
6) Test RMSE: 0.056
Epoch 1/10
586/586 [==============================] - 1s 820us/step - loss: 0.0641
Epoch 2/10
586/586 [==============================] - 0s 830us/step - loss: 0.0143
Epoch 3/10
586/586 [==============================] - 0s 823us/step - loss: 0.0136
Epoch 4/10
586/586 [==============================] - 1s 853us/step - loss: 0.0131
Epoch 5/10
586/586 [==============================] - 1s 879us/step - loss: 0.0128
Epoch 6/10
586/586 [==============================] - 1s 908us/step - loss: 0.0126
Epoch 7/10
586/586 [==============================] - 1s 902us/step - loss: 0.0125
Epoch 8/10
586/586 [==============================] - 1s 883us/step - loss: 0.0124
Epoch 9/10
586/586 [==============================] - 0s 850us/step - loss: 0.0123
Epoch 10/10
586/586 [==============================] - 1s 856us/step - loss: 0.0122
7) Test RMSE: 0.055
Epoch 1/10
586/586 [==============================] - 1s 820us/step - loss: 0.0353
Epoch 2/10
586/586 [==============================] - 0s 836us/step - loss: 0.0174
Epoch 3/10
586/586 [==============================] - 0s 840us/step - loss: 0.0154
Epoch 4/10
586/586 [==============================] - 1s 1ms/step - loss: 0.0142
Epoch 5/10
586/586 [==============================] - 1s 858us/step - loss: 0.0134
Epoch 6/10
586/586 [==============================] - 1s 885us/step - loss: 0.0128
Epoch 7/10
586/586 [==============================] - 0s 853us/step - loss: 0.0124
Epoch 8/10
586/586 [==============================] - 0s 840us/step - loss: 0.0121
Epoch 9/10
586/586 [==============================] - 1s 878us/step - loss: 0.0119
Epoch 10/10
586/586 [==============================] - 1s 882us/step - loss: 0.0118
8) Test RMSE: 0.055
              1         3         5
count  8.000000  8.000000  8.000000
mean   0.055685  0.055600  0.055370
std    0.000123  0.000298  0.000284
min    0.055516  0.055283  0.054911
25%    0.055576  0.055290  0.055196
50%    0.055726  0.055619  0.055360
75%    0.055796  0.055793  0.055535
max    0.055796  0.056043  0.055786

Checking Overfitting

The author plots the test and train RMSE for each of the 10 runs.

In [ ]:
# run diagnostic experiments
def run():
	# config
	repeats = 10
	n_batch = 16
	n_epochs = 20
	n_neurons = 2
	n_lag = 3
    
	series = yieldsp
	raw_values = series.values
	diff_values = difference(raw_values, 1)
    
	supervised = timeseries_to_supervised(diff_values, n_lag)
	supervised_values = supervised.values[n_lag:,:]
    
	train, test = supervised_values[0:-365], supervised_values[-365:]
	scaler, train_scaled, test_scaled = scale(train, test)
    train_trimmed = train_scaled[2:, :]
    
	for i in range(repeats):
		history = fit(train_trimmed, test_scaled, raw_values, scaler, n_batch, n_epochs, n_neurons)
		pyplot.plot(history['train'], color='blue')
		pyplot.plot(history['test'], color='orange')
        print('%d) TrainRMSE=%f, TestRMSE=%f' % (i, history['train'].iloc[-1], history['test'].iloc[-1]))
	pyplot.savefig('diagnostic_neurons_lag.png')
    
run()    
0) TrainRMSE=0.076628, TestRMSE=0.055795
1) TrainRMSE=0.076689, TestRMSE=0.055427
2) TrainRMSE=0.076887, TestRMSE=0.055631
3) TrainRMSE=0.076407, TestRMSE=0.055607
4) TrainRMSE=0.076864, TestRMSE=0.055795
5) TrainRMSE=0.076524, TestRMSE=0.055239
6) TrainRMSE=0.076748, TestRMSE=0.055424
7) TrainRMSE=0.076738, TestRMSE=0.055499
8) TrainRMSE=0.076679, TestRMSE=0.055709
9) TrainRMSE=0.076517, TestRMSE=0.055330

Vanilla LSTMs

The author has constructed a multi-layered LSTM network to forecast the test set values of the yield spread, given the history of observations in the training set.

Pre-processing and feature engineering

In [ ]:
dataset = yieldsp['yieldsp'].values 
dataset = dataset.astype('float32') 
dataset = np.reshape(dataset, (-1, 1)) 
In [ ]:
dataset.shape
Out[ ]:
(9741, 1)

Feature Normalization

As LSTMs are sensitive to the scale of the input data, particularly when we invoke the sigmoid or the hyperbolic tangent activation functions, we need to normalized to the range (0,1).

In [ ]:
scaler = MinMaxScaler(feature_range=(0, 1)) 
dataset = scaler.fit_transform(dataset)
In [ ]:
train, test = np.split(yieldsp, [int(.95 *len(yieldsp))])
train.shape
Out[ ]:
(9253, 1)

Building the LSTM Model

I have created a class to easily model LSTMs.

Inspiration

In [ ]:
from keras.models import Sequential
from keras.layers import LSTM, Dense

class DeepModelTS():
    """
    A class to create a deep time series model
    """
    def __init__(
        self, 
        data: pd.DataFrame, 
        Y_var: str,
        lag: int,
        LSTM_layer_depth: int, 
        epochs=10, 
        batch_size=256,
        train_test_split=0,
        patience = 5
    ):

        self.data = data 
        self.Y_var = Y_var 
        self.lag = lag 
        self.LSTM_layer_depth = LSTM_layer_depth
        self.batch_size = batch_size
        self.epochs = epochs
        self.patience = patience
        self.train_test_split = train_test_split

    @staticmethod
    def create_X_Y(ts: list, lag: int) -> tuple:
        """
        A method to create X and Y matrix from a time series list for the training of 
        deep learning models 
        """
        X, Y = [], []

        if len(ts) - lag <= 0:
            X.append(ts)
        else:
            for i in range(len(ts) - lag):
                Y.append(ts[i + lag])
                X.append(ts[i:(i + lag)])

        X, Y = np.array(X), np.array(Y)

        # Reshaping the X array to an LSTM input shape 
        X = np.reshape(X, (X.shape[0], X.shape[1], 1))

        return X, Y         

    def create_data_for_NN(self,use_last_n=None):
        """
        A method to create data for the LSTM model
        """
        # Extracting the dependent variable we want to model/forecast
        y = self.data[self.Y_var].tolist()

        # Subseting the time series if needed
        if use_last_n is not None:
            y = y[-use_last_n:]

        # The X matrix will hold the lags of Y 
        X, Y = self.create_X_Y(y, self.lag)

        # Creating training and test sets 
        X_train = X
        X_test = []

        Y_train = Y
        Y_test = []

        if self.train_test_split > 0:
            index = round(len(X) * self.train_test_split)
            X_train = X[:(len(X) - index)]
            X_test = X[-index:]     
            
            Y_train = Y[:(len(X) - index)]
            Y_test = Y[-index:]

        return X_train, X_test, Y_train, Y_test

    def LSTModel(self):
        """
        A method to fit the LSTM model 
        """
        # Getting the data 
        X_train, X_test, Y_train, Y_test = self.create_data_for_NN()

        # Defining the model
        model = Sequential()
        model.add(LSTM(self.LSTM_layer_depth, activation='relu', input_shape=(self.lag, 1)))
        model.add(Dense(1))
        model.compile(optimizer='adam', loss='mse')

        # Fitting the model 
        model.fit(X_train, Y_train, 
                  self.batch_size,
                  self.epochs, 
                  validation_data=(X_test, Y_test), 
                  callbacks = [EarlyStopping(monitor='val_loss', patience= self.patience)], 
                  verbose=1, shuffle=False)

        # Saving the model to the class 
        self.model = model

        return model

    def predict(self) -> list:
        """
        A method to predict using the test data used in creating the class
        """
        yhat = []

        if(self.train_test_split > 0):
        
            # Getting the last n time series 
            _, X_test, _, _ = self.create_data_for_NN()        

            # Making the prediction list 
            yhat = [y[0] for y in self.model.predict(X_test)]

        return yhat

    def predict_n_ahead(self, n_ahead: int):
        """
        A method to predict n time steps ahead
        """    
        X, _, _, _ = self.create_data_for_NN(use_last_n=self.lag)        

        # Making the prediction list 
        yhat = []

        for _ in range(n_ahead):
            # Making the prediction
            fc = self.model.predict(X)
            yhat.append(fc)

            # Creating a new input matrix for forecasting
            X = np.append(X, fc)

            # Ommiting the first variable
            X = np.delete(X, 0)

            # Reshaping for the next iteration
            X = np.reshape(X, (1, len(X), 1))
  • the model architecture consists of only 1 LSTM layer with 50 neurons
  • Dropout) of 0.2 is used; which randomly ignores 20% of the neurons in each training epoch, helping regularize and reduce overfitting.
  • the MSE loss function is used
  • the ADAM optimizer is used
  • the model has been trained for 200 epochs.
In [ ]:
# Training the model with more lag periods increases the training time:
deep_learner = DeepModelTS(data = train, Y_var = 'yieldsp',  lag = 50, 
                           LSTM_layer_depth = 50,  epochs = 1000 ,batch_size = 64,
                           train_test_split = 0.05, patience = 200)

model = deep_learner.LSTModel()
Epoch 1/1000
137/137 [==============================] - 6s 36ms/step - loss: 8.6993 - val_loss: 0.4577
Epoch 2/1000
137/137 [==============================] - 3s 24ms/step - loss: 28.1427 - val_loss: 0.7448
Epoch 3/1000
137/137 [==============================] - 4s 26ms/step - loss: 2.9275 - val_loss: 0.5468
Epoch 4/1000
137/137 [==============================] - 3s 23ms/step - loss: 2.5108 - val_loss: 0.3654
Epoch 5/1000
137/137 [==============================] - 3s 23ms/step - loss: 1.9275 - val_loss: 0.1838
Epoch 6/1000
137/137 [==============================] - 3s 23ms/step - loss: 1.1042 - val_loss: 0.0121
Epoch 7/1000
137/137 [==============================] - 3s 24ms/step - loss: 46.8189 - val_loss: 0.2534
Epoch 8/1000
137/137 [==============================] - 3s 23ms/step - loss: 1.8109 - val_loss: 0.1778
Epoch 9/1000
137/137 [==============================] - 3s 24ms/step - loss: 1.5581 - val_loss: 0.1190
Epoch 10/1000
137/137 [==============================] - 3s 25ms/step - loss: 1.3319 - val_loss: 0.0766
Epoch 11/1000
137/137 [==============================] - 3s 24ms/step - loss: 1.1306 - val_loss: 0.0490
Epoch 12/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.9512 - val_loss: 0.0337
Epoch 13/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.7921 - val_loss: 0.0288
Epoch 14/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.6545 - val_loss: 0.0320
Epoch 15/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.5324 - val_loss: 0.0359
Epoch 16/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.4234 - val_loss: 0.0333
Epoch 17/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.3305 - val_loss: 0.0123
Epoch 18/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.2443 - val_loss: 0.1054
Epoch 19/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.2027 - val_loss: 0.0580
Epoch 20/1000
137/137 [==============================] - 5s 34ms/step - loss: 0.1618 - val_loss: 0.0138
Epoch 21/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.1172 - val_loss: 0.0047
Epoch 22/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0868 - val_loss: 0.0043
Epoch 23/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0650 - val_loss: 0.0040
Epoch 24/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0504 - val_loss: 0.0039
Epoch 25/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0411 - val_loss: 0.0041
Epoch 26/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0353 - val_loss: 0.0043
Epoch 27/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0316 - val_loss: 0.0044
Epoch 28/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0293 - val_loss: 0.0044
Epoch 29/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0278 - val_loss: 0.0043
Epoch 30/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0265 - val_loss: 0.0047
Epoch 31/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0256 - val_loss: 0.0054
Epoch 32/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0244 - val_loss: 0.0057
Epoch 33/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0231 - val_loss: 0.0049
Epoch 34/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0219 - val_loss: 0.0041
Epoch 35/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0212 - val_loss: 0.0039
Epoch 36/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0212 - val_loss: 0.0038
Epoch 37/1000
137/137 [==============================] - 4s 27ms/step - loss: 0.0210 - val_loss: 0.0033
Epoch 38/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0203 - val_loss: 0.0031
Epoch 39/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0195 - val_loss: 0.0035
Epoch 40/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0191 - val_loss: 0.0037
Epoch 41/1000
137/137 [==============================] - 4s 32ms/step - loss: 0.0187 - val_loss: 0.0036
Epoch 42/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0184 - val_loss: 0.0036
Epoch 43/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0180 - val_loss: 0.0035
Epoch 44/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0177 - val_loss: 0.0035
Epoch 45/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0173 - val_loss: 0.0035
Epoch 46/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0171 - val_loss: 0.0032
Epoch 47/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0168 - val_loss: 0.0033
Epoch 48/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0165 - val_loss: 0.0033
Epoch 49/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0163 - val_loss: 0.0033
Epoch 50/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0160 - val_loss: 0.0034
Epoch 51/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0158 - val_loss: 0.0034
Epoch 52/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0156 - val_loss: 0.0036
Epoch 53/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0155 - val_loss: 0.0037
Epoch 54/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0153 - val_loss: 0.0036
Epoch 55/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0151 - val_loss: 0.0038
Epoch 56/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0150 - val_loss: 0.0038
Epoch 57/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0148 - val_loss: 0.0039
Epoch 58/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0146 - val_loss: 0.0041
Epoch 59/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0145 - val_loss: 0.0042
Epoch 60/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0144 - val_loss: 0.0043
Epoch 61/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0143 - val_loss: 0.0044
Epoch 62/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0142 - val_loss: 0.0046
Epoch 63/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0141 - val_loss: 0.0046
Epoch 64/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0140 - val_loss: 0.0047
Epoch 65/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0139 - val_loss: 0.0048
Epoch 66/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0137 - val_loss: 0.0049
Epoch 67/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0136 - val_loss: 0.0050
Epoch 68/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0135 - val_loss: 0.0051
Epoch 69/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0134 - val_loss: 0.0051
Epoch 70/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0133 - val_loss: 0.0052
Epoch 71/1000
137/137 [==============================] - 4s 32ms/step - loss: 0.0132 - val_loss: 0.0052
Epoch 72/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0131 - val_loss: 0.0053
Epoch 73/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0130 - val_loss: 0.0052
Epoch 74/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0129 - val_loss: 0.0052
Epoch 75/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0127 - val_loss: 0.0052
Epoch 76/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0126 - val_loss: 0.0052
Epoch 77/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0125 - val_loss: 0.0053
Epoch 78/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0124 - val_loss: 0.0054
Epoch 79/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0122 - val_loss: 0.0055
Epoch 80/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0121 - val_loss: 0.0057
Epoch 81/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0120 - val_loss: 0.0058
Epoch 82/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0119 - val_loss: 0.0058
Epoch 83/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0119 - val_loss: 0.0058
Epoch 84/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0118 - val_loss: 0.0057
Epoch 85/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0117 - val_loss: 0.0057
Epoch 86/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0116 - val_loss: 0.0056
Epoch 87/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0115 - val_loss: 0.0056
Epoch 88/1000
137/137 [==============================] - 4s 30ms/step - loss: 0.0115 - val_loss: 0.0055
Epoch 89/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0113 - val_loss: 0.0054
Epoch 90/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0112 - val_loss: 0.0054
Epoch 91/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0111 - val_loss: 0.0053
Epoch 92/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0110 - val_loss: 0.0053
Epoch 93/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0109 - val_loss: 0.0052
Epoch 94/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0108 - val_loss: 0.0051
Epoch 95/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0107 - val_loss: 0.0050
Epoch 96/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0106 - val_loss: 0.0051
Epoch 97/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0106 - val_loss: 0.0049
Epoch 98/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0105 - val_loss: 0.0050
Epoch 99/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0105 - val_loss: 0.0049
Epoch 100/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0104 - val_loss: 0.0048
Epoch 101/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0103 - val_loss: 0.0046
Epoch 102/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0102 - val_loss: 0.0047
Epoch 103/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0102 - val_loss: 0.0047
Epoch 104/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0101 - val_loss: 0.0046
Epoch 105/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0101 - val_loss: 0.0047
Epoch 106/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0101 - val_loss: 0.0042
Epoch 107/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0099 - val_loss: 0.0046
Epoch 108/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0100 - val_loss: 0.0046
Epoch 109/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0099 - val_loss: 0.0045
Epoch 110/1000
137/137 [==============================] - 4s 30ms/step - loss: 0.0098 - val_loss: 0.0048
Epoch 111/1000
137/137 [==============================] - 4s 27ms/step - loss: 0.0098 - val_loss: 0.0048
Epoch 112/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0098 - val_loss: 0.0044
Epoch 113/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0097 - val_loss: 0.0047
Epoch 114/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0097 - val_loss: 0.0047
Epoch 115/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0096 - val_loss: 0.0047
Epoch 116/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0096 - val_loss: 0.0049
Epoch 117/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0096 - val_loss: 0.0048
Epoch 118/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0096 - val_loss: 0.0051
Epoch 119/1000
137/137 [==============================] - 4s 27ms/step - loss: 0.0097 - val_loss: 0.0046
Epoch 120/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0096 - val_loss: 0.0052
Epoch 121/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0096 - val_loss: 0.0047
Epoch 122/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0095 - val_loss: 0.0050
Epoch 123/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0095 - val_loss: 0.0047
Epoch 124/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0094 - val_loss: 0.0036
Epoch 125/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0091 - val_loss: 0.0063
Epoch 126/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0097 - val_loss: 0.0046
Epoch 127/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0095 - val_loss: 0.0063
Epoch 128/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0093 - val_loss: 0.0052
Epoch 129/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0094 - val_loss: 0.0051
Epoch 130/1000
137/137 [==============================] - 4s 30ms/step - loss: 0.0093 - val_loss: 0.0054
Epoch 131/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0093 - val_loss: 0.0051
Epoch 132/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0093 - val_loss: 0.0050
Epoch 133/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0093 - val_loss: 0.0051
Epoch 134/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0092 - val_loss: 0.0050
Epoch 135/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0092 - val_loss: 0.0050
Epoch 136/1000
137/137 [==============================] - 4s 28ms/step - loss: 0.0092 - val_loss: 0.0051
Epoch 137/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0092 - val_loss: 0.0048
Epoch 138/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0091 - val_loss: 0.0051
Epoch 139/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0091 - val_loss: 0.0046
Epoch 140/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0091 - val_loss: 0.0053
Epoch 141/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0090 - val_loss: 0.0044
Epoch 142/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0090 - val_loss: 0.0053
Epoch 143/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0090 - val_loss: 0.0047
Epoch 144/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0090 - val_loss: 0.0050
Epoch 145/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0090 - val_loss: 0.0044
Epoch 146/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0089 - val_loss: 0.0048
Epoch 147/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0089 - val_loss: 0.0047
Epoch 148/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0089 - val_loss: 0.0048
Epoch 149/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0088 - val_loss: 0.0040
Epoch 150/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0090 - val_loss: 0.0055
Epoch 151/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0088 - val_loss: 0.0037
Epoch 152/1000
137/137 [==============================] - 5s 33ms/step - loss: 0.0088 - val_loss: 0.0043
Epoch 153/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0089 - val_loss: 0.0043
Epoch 154/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0088 - val_loss: 0.0045
Epoch 155/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0088 - val_loss: 0.0045
Epoch 156/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0087 - val_loss: 0.0028
Epoch 157/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0086 - val_loss: 0.0045
Epoch 158/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0089 - val_loss: 0.0050
Epoch 159/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0088 - val_loss: 0.0036
Epoch 160/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0087 - val_loss: 0.0041
Epoch 161/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0088 - val_loss: 0.0046
Epoch 162/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0088 - val_loss: 0.0041
Epoch 163/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0088 - val_loss: 0.0033
Epoch 164/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0089 - val_loss: 0.0047
Epoch 165/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0089 - val_loss: 0.0047
Epoch 166/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0089 - val_loss: 0.0043
Epoch 167/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0087 - val_loss: 0.0038
Epoch 168/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0086 - val_loss: 0.0040
Epoch 169/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0087 - val_loss: 0.0039
Epoch 170/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0086 - val_loss: 0.0037
Epoch 171/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0086 - val_loss: 0.0037
Epoch 172/1000
137/137 [==============================] - 4s 29ms/step - loss: 0.0086 - val_loss: 0.0039
Epoch 173/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0087 - val_loss: 0.0038
Epoch 174/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0086 - val_loss: 0.0040
Epoch 175/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0086 - val_loss: 0.0038
Epoch 176/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0086 - val_loss: 0.0037
Epoch 177/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0086 - val_loss: 0.0037
Epoch 178/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0085 - val_loss: 0.0039
Epoch 179/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0087 - val_loss: 0.0039
Epoch 180/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0086 - val_loss: 0.0039
Epoch 181/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0086 - val_loss: 0.0035
Epoch 182/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0085 - val_loss: 0.0040
Epoch 183/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0087 - val_loss: 0.0038
Epoch 184/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0086 - val_loss: 0.0038
Epoch 185/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0086 - val_loss: 0.0038
Epoch 186/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0086 - val_loss: 0.0035
Epoch 187/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0085 - val_loss: 0.0037
Epoch 188/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0086 - val_loss: 0.0034
Epoch 189/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0085 - val_loss: 0.0034
Epoch 190/1000
137/137 [==============================] - 4s 30ms/step - loss: 0.0086 - val_loss: 0.0043
Epoch 191/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0086 - val_loss: 0.0035
Epoch 192/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0086 - val_loss: 0.0034
Epoch 193/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0085 - val_loss: 0.0034
Epoch 194/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0085 - val_loss: 0.0034
Epoch 195/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0085 - val_loss: 0.0028
Epoch 196/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0099 - val_loss: 0.0045
Epoch 197/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0088 - val_loss: 0.0042
Epoch 198/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0086 - val_loss: 0.0030
Epoch 199/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0084 - val_loss: 0.0029
Epoch 200/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0084 - val_loss: 0.0032
Epoch 201/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0084 - val_loss: 0.0033
Epoch 202/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0085 - val_loss: 0.0036
Epoch 203/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0085 - val_loss: 0.0035
Epoch 204/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0085 - val_loss: 0.0033
Epoch 205/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0084 - val_loss: 0.0033
Epoch 206/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0085 - val_loss: 0.0034
Epoch 207/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0084 - val_loss: 0.0033
Epoch 208/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0084 - val_loss: 0.0032
Epoch 209/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0084 - val_loss: 0.0031
Epoch 210/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0085 - val_loss: 0.0031
Epoch 211/1000
137/137 [==============================] - 4s 31ms/step - loss: 0.0084 - val_loss: 0.0031
Epoch 212/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0084 - val_loss: 0.0033
Epoch 213/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0085 - val_loss: 0.0035
Epoch 214/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0085 - val_loss: 0.0027
Epoch 215/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0088 - val_loss: 0.0035
Epoch 216/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0085 - val_loss: 0.0037
Epoch 217/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0086 - val_loss: 0.0031
Epoch 218/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0084 - val_loss: 0.0031
Epoch 219/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0084 - val_loss: 0.0032
Epoch 220/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0084 - val_loss: 0.0032
Epoch 221/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0084 - val_loss: 0.0032
Epoch 222/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0084 - val_loss: 0.0031
Epoch 223/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0084 - val_loss: 0.0032
Epoch 224/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0084 - val_loss: 0.0031
Epoch 225/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0084 - val_loss: 0.0030
Epoch 226/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0084 - val_loss: 0.0030
Epoch 227/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0084 - val_loss: 0.0033
Epoch 228/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0084 - val_loss: 0.0031
Epoch 229/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0084 - val_loss: 0.0031
Epoch 230/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0084 - val_loss: 0.0031
Epoch 231/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0084 - val_loss: 0.0031
Epoch 232/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0084 - val_loss: 0.0031
Epoch 233/1000
137/137 [==============================] - 4s 31ms/step - loss: 0.0084 - val_loss: 0.0031
Epoch 234/1000
137/137 [==============================] - 4s 27ms/step - loss: 0.0084 - val_loss: 0.0032
Epoch 235/1000
137/137 [==============================] - 4s 27ms/step - loss: 0.0084 - val_loss: 0.0031
Epoch 236/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0084 - val_loss: 0.0031
Epoch 237/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0084 - val_loss: 0.0030
Epoch 238/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0084 - val_loss: 0.0030
Epoch 239/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0084 - val_loss: 0.0031
Epoch 240/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0084 - val_loss: 0.0029
Epoch 241/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0084 - val_loss: 0.0030
Epoch 242/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0084 - val_loss: 0.0030
Epoch 243/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0084 - val_loss: 0.0030
Epoch 244/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0084 - val_loss: 0.0029
Epoch 245/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0084 - val_loss: 0.0029
Epoch 246/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0084 - val_loss: 0.0029
Epoch 247/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0084 - val_loss: 0.0025
Epoch 248/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0083 - val_loss: 0.0027
Epoch 249/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0084 - val_loss: 0.0032
Epoch 250/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0084 - val_loss: 0.0030
Epoch 251/1000
137/137 [==============================] - 4s 27ms/step - loss: 0.0084 - val_loss: 0.0029
Epoch 252/1000
137/137 [==============================] - 4s 29ms/step - loss: 0.0084 - val_loss: 0.0029
Epoch 253/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0083 - val_loss: 0.0028
Epoch 254/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0083 - val_loss: 0.0027
Epoch 255/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0083 - val_loss: 0.0029
Epoch 256/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0083 - val_loss: 0.0028
Epoch 257/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0083 - val_loss: 0.0028
Epoch 258/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0083 - val_loss: 0.0029
Epoch 259/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0083 - val_loss: 0.0028
Epoch 260/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0083 - val_loss: 0.0028
Epoch 261/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0083 - val_loss: 0.0028
Epoch 262/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0082 - val_loss: 0.0027
Epoch 263/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0083 - val_loss: 0.0029
Epoch 264/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0083 - val_loss: 0.0027
Epoch 265/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0083 - val_loss: 0.0027
Epoch 266/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0083 - val_loss: 0.0028
Epoch 267/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0082 - val_loss: 0.0030
Epoch 268/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0083 - val_loss: 0.0026
Epoch 269/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0083 - val_loss: 0.0027
Epoch 270/1000
137/137 [==============================] - 4s 29ms/step - loss: 0.0083 - val_loss: 0.0028
Epoch 271/1000
137/137 [==============================] - 4s 27ms/step - loss: 0.0082 - val_loss: 0.0027
Epoch 272/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0082 - val_loss: 0.0026
Epoch 273/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0083 - val_loss: 0.0026
Epoch 274/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0082 - val_loss: 0.0028
Epoch 275/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0083 - val_loss: 0.0026
Epoch 276/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0083 - val_loss: 0.0026
Epoch 277/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0082 - val_loss: 0.0028
Epoch 278/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0083 - val_loss: 0.0025
Epoch 279/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0082 - val_loss: 0.0026
Epoch 280/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0082 - val_loss: 0.0024
Epoch 281/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0082 - val_loss: 0.0026
Epoch 282/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0081 - val_loss: 0.0025
Epoch 283/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0082 - val_loss: 0.0027
Epoch 284/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0082 - val_loss: 0.0025
Epoch 285/1000
137/137 [==============================] - 4s 27ms/step - loss: 0.0082 - val_loss: 0.0023
Epoch 286/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0082 - val_loss: 0.0028
Epoch 287/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0082 - val_loss: 0.0024
Epoch 288/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0081 - val_loss: 0.0025
Epoch 289/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0081 - val_loss: 0.0026
Epoch 290/1000
137/137 [==============================] - 4s 31ms/step - loss: 0.0081 - val_loss: 0.0025
Epoch 291/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0081 - val_loss: 0.0024
Epoch 292/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0081 - val_loss: 0.0028
Epoch 293/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0081 - val_loss: 0.0025
Epoch 294/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0082 - val_loss: 0.0027
Epoch 295/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0080 - val_loss: 0.0024
Epoch 296/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0081 - val_loss: 0.0026
Epoch 297/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0081 - val_loss: 0.0026
Epoch 298/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0081 - val_loss: 0.0025
Epoch 299/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0079 - val_loss: 0.0024
Epoch 300/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0080 - val_loss: 0.0025
Epoch 301/1000
137/137 [==============================] - 4s 27ms/step - loss: 0.0080 - val_loss: 0.0026
Epoch 302/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0079 - val_loss: 0.0025
Epoch 303/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0079 - val_loss: 0.0025
Epoch 304/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0079 - val_loss: 0.0028
Epoch 305/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0080 - val_loss: 0.0028
Epoch 306/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0079 - val_loss: 0.0025
Epoch 307/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0080 - val_loss: 0.0026
Epoch 308/1000
137/137 [==============================] - 4s 31ms/step - loss: 0.0080 - val_loss: 0.0026
Epoch 309/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0080 - val_loss: 0.0025
Epoch 310/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0081 - val_loss: 0.0024
Epoch 311/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0084 - val_loss: 0.0021
Epoch 312/1000
137/137 [==============================] - 4s 27ms/step - loss: 0.0082 - val_loss: 0.0029
Epoch 313/1000
137/137 [==============================] - 4s 27ms/step - loss: 0.0080 - val_loss: 0.0021
Epoch 314/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0079 - val_loss: 0.0027
Epoch 315/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0079 - val_loss: 0.0027
Epoch 316/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0081 - val_loss: 0.0033
Epoch 317/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0080 - val_loss: 0.0020
Epoch 318/1000
137/137 [==============================] - 4s 27ms/step - loss: 0.0080 - val_loss: 0.0024
Epoch 319/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0080 - val_loss: 0.0020
Epoch 320/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0080 - val_loss: 0.0025
Epoch 321/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0079 - val_loss: 0.0020
Epoch 322/1000
137/137 [==============================] - 4s 27ms/step - loss: 0.0080 - val_loss: 0.0027
Epoch 323/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0079 - val_loss: 0.0021
Epoch 324/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0079 - val_loss: 0.0026
Epoch 325/1000
137/137 [==============================] - 3s 26ms/step - loss: 0.0079 - val_loss: 0.0021
Epoch 326/1000
137/137 [==============================] - 4s 30ms/step - loss: 0.0079 - val_loss: 0.0026
Epoch 327/1000
137/137 [==============================] - 4s 28ms/step - loss: 0.0079 - val_loss: 0.0022
Epoch 328/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0080 - val_loss: 0.0027
Epoch 329/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0079 - val_loss: 0.0022
Epoch 330/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0080 - val_loss: 0.0026
Epoch 331/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0079 - val_loss: 0.0031
Epoch 332/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0081 - val_loss: 0.0028
Epoch 333/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0078 - val_loss: 0.0023
Epoch 334/1000
137/137 [==============================] - 4s 27ms/step - loss: 0.0078 - val_loss: 0.0024
Epoch 335/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0077 - val_loss: 0.0023
Epoch 336/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0078 - val_loss: 0.0022
Epoch 337/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0077 - val_loss: 0.0025
Epoch 338/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0077 - val_loss: 0.0024
Epoch 339/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0077 - val_loss: 0.0026
Epoch 340/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0079 - val_loss: 0.0024
Epoch 341/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0078 - val_loss: 0.0025
Epoch 342/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0077 - val_loss: 0.0021
Epoch 343/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0077 - val_loss: 0.0025
Epoch 344/1000
137/137 [==============================] - 4s 30ms/step - loss: 0.0077 - val_loss: 0.0023
Epoch 345/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0077 - val_loss: 0.0024
Epoch 346/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0078 - val_loss: 0.0024
Epoch 347/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0084 - val_loss: 0.0024
Epoch 348/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0082 - val_loss: 0.0027
Epoch 349/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0080 - val_loss: 0.0021
Epoch 350/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0079 - val_loss: 0.0023
Epoch 351/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0077 - val_loss: 0.0022
Epoch 352/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0077 - val_loss: 0.0025
Epoch 353/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0077 - val_loss: 0.0022
Epoch 354/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0076 - val_loss: 0.0025
Epoch 355/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0076 - val_loss: 0.0025
Epoch 356/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0110 - val_loss: 0.0016
Epoch 357/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0082 - val_loss: 0.0015
Epoch 358/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0082 - val_loss: 0.0015
Epoch 359/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0081 - val_loss: 0.0015
Epoch 360/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0081 - val_loss: 0.0016
Epoch 361/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0081 - val_loss: 0.0016
Epoch 362/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0081 - val_loss: 0.0017
Epoch 363/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0081 - val_loss: 0.0018
Epoch 364/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0081 - val_loss: 0.0018
Epoch 365/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0081 - val_loss: 0.0019
Epoch 366/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0081 - val_loss: 0.0020
Epoch 367/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0081 - val_loss: 0.0020
Epoch 368/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0081 - val_loss: 0.0021
Epoch 369/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0081 - val_loss: 0.0021
Epoch 370/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0081 - val_loss: 0.0022
Epoch 371/1000
137/137 [==============================] - 4s 32ms/step - loss: 0.0081 - val_loss: 0.0023
Epoch 372/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0081 - val_loss: 0.0023
Epoch 373/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0081 - val_loss: 0.0024
Epoch 374/1000
137/137 [==============================] - 3s 26ms/step - loss: 0.0081 - val_loss: 0.0029
Epoch 375/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0082 - val_loss: 0.0021
Epoch 376/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0081 - val_loss: 0.0024
Epoch 377/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0081 - val_loss: 0.0025
Epoch 378/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0081 - val_loss: 0.0024
Epoch 379/1000
137/137 [==============================] - 3s 24ms/step - loss: 0.0081 - val_loss: 0.0025
Epoch 380/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0081 - val_loss: 0.0024
Epoch 381/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0081 - val_loss: 0.0025
Epoch 382/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0081 - val_loss: 0.0026
Epoch 383/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0081 - val_loss: 0.0022
Epoch 384/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0080 - val_loss: 0.0026
Epoch 385/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0081 - val_loss: 0.0024
Epoch 386/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0081 - val_loss: 0.0025
Epoch 387/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0081 - val_loss: 0.0024
Epoch 388/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0081 - val_loss: 0.0026
Epoch 389/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0081 - val_loss: 0.0026
Epoch 390/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0081 - val_loss: 0.0026
Epoch 391/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0081 - val_loss: 0.0026
Epoch 392/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0081 - val_loss: 0.0025
Epoch 393/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0080 - val_loss: 0.0028
Epoch 394/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0081 - val_loss: 0.0023
Epoch 395/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0080 - val_loss: 0.0027
Epoch 396/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0080 - val_loss: 0.0023
Epoch 397/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0080 - val_loss: 0.0025
Epoch 398/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0079 - val_loss: 0.0026
Epoch 399/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0079 - val_loss: 0.0023
Epoch 400/1000
137/137 [==============================] - 4s 29ms/step - loss: 0.0079 - val_loss: 0.0025
Epoch 401/1000
137/137 [==============================] - 4s 32ms/step - loss: 0.0079 - val_loss: 0.0023
Epoch 402/1000
137/137 [==============================] - 4s 28ms/step - loss: 0.0079 - val_loss: 0.0027
Epoch 403/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0079 - val_loss: 0.0021
Epoch 404/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0079 - val_loss: 0.0022
Epoch 405/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0077 - val_loss: 0.0027
Epoch 406/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0078 - val_loss: 0.0024
Epoch 407/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0077 - val_loss: 0.0024
Epoch 408/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0077 - val_loss: 0.0025
Epoch 409/1000
137/137 [==============================] - 4s 27ms/step - loss: 0.0078 - val_loss: 0.0024
Epoch 410/1000
137/137 [==============================] - 4s 27ms/step - loss: 0.0078 - val_loss: 0.0025
Epoch 411/1000
137/137 [==============================] - 4s 27ms/step - loss: 0.0078 - val_loss: 0.0021
Epoch 412/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0077 - val_loss: 0.0026
Epoch 413/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0077 - val_loss: 0.0022
Epoch 414/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0077 - val_loss: 0.0023
Epoch 415/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0077 - val_loss: 0.0023
Epoch 416/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0077 - val_loss: 0.0022
Epoch 417/1000
137/137 [==============================] - 4s 28ms/step - loss: 0.0077 - val_loss: 0.0023
Epoch 418/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0077 - val_loss: 0.0026
Epoch 419/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0078 - val_loss: 0.0019
Epoch 420/1000
137/137 [==============================] - 5s 34ms/step - loss: 0.0078 - val_loss: 0.0026
Epoch 421/1000
137/137 [==============================] - 4s 27ms/step - loss: 0.0075 - val_loss: 0.0019
Epoch 422/1000
137/137 [==============================] - 4s 27ms/step - loss: 0.0076 - val_loss: 0.0022
Epoch 423/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0075 - val_loss: 0.0022
Epoch 424/1000
137/137 [==============================] - 4s 27ms/step - loss: 0.0077 - val_loss: 0.0021
Epoch 425/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0077 - val_loss: 0.0024
Epoch 426/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0076 - val_loss: 0.0020
Epoch 427/1000
137/137 [==============================] - 3s 26ms/step - loss: 0.0080 - val_loss: 0.0024
Epoch 428/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0078 - val_loss: 0.0019
Epoch 429/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0076 - val_loss: 0.0024
Epoch 430/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0077 - val_loss: 0.0020
Epoch 431/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0080 - val_loss: 0.0027
Epoch 432/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0078 - val_loss: 0.0020
Epoch 433/1000
137/137 [==============================] - 4s 28ms/step - loss: 0.0078 - val_loss: 0.0023
Epoch 434/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0076 - val_loss: 0.0022
Epoch 435/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0077 - val_loss: 0.0023
Epoch 436/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0075 - val_loss: 0.0020
Epoch 437/1000
137/137 [==============================] - 5s 34ms/step - loss: 0.0077 - val_loss: 0.0022
Epoch 438/1000
137/137 [==============================] - 4s 27ms/step - loss: 0.0077 - val_loss: 0.0022
Epoch 439/1000
137/137 [==============================] - 4s 27ms/step - loss: 0.0076 - val_loss: 0.0021
Epoch 440/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0075 - val_loss: 0.0021
Epoch 441/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0075 - val_loss: 0.0022
Epoch 442/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0075 - val_loss: 0.0023
Epoch 443/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0075 - val_loss: 0.0019
Epoch 444/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0075 - val_loss: 0.0022
Epoch 445/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0076 - val_loss: 0.0024
Epoch 446/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0076 - val_loss: 0.0020
Epoch 447/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0074 - val_loss: 0.0022
Epoch 448/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0077 - val_loss: 0.0023
Epoch 449/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0081 - val_loss: 0.0023
Epoch 450/1000
137/137 [==============================] - 4s 27ms/step - loss: 0.0080 - val_loss: 0.0025
Epoch 451/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0077 - val_loss: 0.0020
Epoch 452/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0076 - val_loss: 0.0023
Epoch 453/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0074 - val_loss: 0.0019
Epoch 454/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0074 - val_loss: 0.0023
Epoch 455/1000
137/137 [==============================] - 5s 33ms/step - loss: 0.0075 - val_loss: 0.0020
Epoch 456/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0075 - val_loss: 0.0024
Epoch 457/1000
137/137 [==============================] - 4s 27ms/step - loss: 0.0075 - val_loss: 0.0019
Epoch 458/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0077 - val_loss: 0.0024
Epoch 459/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0075 - val_loss: 0.0019
Epoch 460/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0073 - val_loss: 0.0025
Epoch 461/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0076 - val_loss: 0.0024
Epoch 462/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0075 - val_loss: 0.0017
Epoch 463/1000
137/137 [==============================] - 3s 26ms/step - loss: 0.0073 - val_loss: 0.0020
Epoch 464/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0073 - val_loss: 0.0020
Epoch 465/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0074 - val_loss: 0.0021
Epoch 466/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0074 - val_loss: 0.0019
Epoch 467/1000
137/137 [==============================] - 4s 27ms/step - loss: 0.0074 - val_loss: 0.0023
Epoch 468/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0077 - val_loss: 0.0028
Epoch 469/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0080 - val_loss: 0.0018
Epoch 470/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0076 - val_loss: 0.0020
Epoch 471/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0075 - val_loss: 0.0020
Epoch 472/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0077 - val_loss: 0.0021
Epoch 473/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0080 - val_loss: 0.0018
Epoch 474/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0076 - val_loss: 0.0023
Epoch 475/1000
137/137 [==============================] - 5s 32ms/step - loss: 0.0074 - val_loss: 0.0021
Epoch 476/1000
137/137 [==============================] - 4s 27ms/step - loss: 0.0073 - val_loss: 0.0021
Epoch 477/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0073 - val_loss: 0.0022
Epoch 478/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0073 - val_loss: 0.0022
Epoch 479/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0075 - val_loss: 0.0020
Epoch 480/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0073 - val_loss: 0.0021
Epoch 481/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0075 - val_loss: 0.0026
Epoch 482/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0075 - val_loss: 0.0019
Epoch 483/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0076 - val_loss: 0.0023
Epoch 484/1000
137/137 [==============================] - 4s 28ms/step - loss: 0.0074 - val_loss: 0.0023
Epoch 485/1000
137/137 [==============================] - 4s 27ms/step - loss: 0.0073 - val_loss: 0.0021
Epoch 486/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0074 - val_loss: 0.0022
Epoch 487/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0073 - val_loss: 0.0020
Epoch 488/1000
137/137 [==============================] - 3s 26ms/step - loss: 0.0072 - val_loss: 0.0021
Epoch 489/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0072 - val_loss: 0.0023
Epoch 490/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0071 - val_loss: 0.0020
Epoch 491/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0073 - val_loss: 0.0022
Epoch 492/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0072 - val_loss: 0.0021
Epoch 493/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0074 - val_loss: 0.0026
Epoch 494/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0072 - val_loss: 0.0017
Epoch 495/1000
137/137 [==============================] - 4s 27ms/step - loss: 0.0070 - val_loss: 0.0019
Epoch 496/1000
137/137 [==============================] - 4s 27ms/step - loss: 0.0074 - val_loss: 0.0024
Epoch 497/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0072 - val_loss: 0.0018
Epoch 498/1000
137/137 [==============================] - 4s 32ms/step - loss: 0.0068 - val_loss: 0.0021
Epoch 499/1000
137/137 [==============================] - 4s 28ms/step - loss: 0.0071 - val_loss: 0.0021
Epoch 500/1000
137/137 [==============================] - 4s 28ms/step - loss: 0.0078 - val_loss: 0.0021
Epoch 501/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0072 - val_loss: 0.0016
Epoch 502/1000
137/137 [==============================] - 4s 26ms/step - loss: 20779.4234 - val_loss: 0.0036
Epoch 503/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0195 - val_loss: 0.0044
Epoch 504/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0168 - val_loss: 0.0032
Epoch 505/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0159 - val_loss: 0.0025
Epoch 506/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0142 - val_loss: 0.0018
Epoch 507/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0182 - val_loss: 0.0020
Epoch 508/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0146 - val_loss: 0.0018
Epoch 509/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0143 - val_loss: 0.0017
Epoch 510/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0140 - val_loss: 0.0017
Epoch 511/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0137 - val_loss: 0.0016
Epoch 512/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0135 - val_loss: 0.0016
Epoch 513/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0133 - val_loss: 0.0016
Epoch 514/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0131 - val_loss: 0.0016
Epoch 515/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0130 - val_loss: 0.0016
Epoch 516/1000
137/137 [==============================] - 4s 32ms/step - loss: 0.0128 - val_loss: 0.0016
Epoch 517/1000
137/137 [==============================] - 4s 28ms/step - loss: 0.0126 - val_loss: 0.0016
Epoch 518/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0125 - val_loss: 0.0016
Epoch 519/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0123 - val_loss: 0.0016
Epoch 520/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0120 - val_loss: 0.0016
Epoch 521/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0118 - val_loss: 0.0016
Epoch 522/1000
137/137 [==============================] - 4s 27ms/step - loss: 0.0114 - val_loss: 0.0016
Epoch 523/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0109 - val_loss: 0.0016
Epoch 524/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0109 - val_loss: 0.0016
Epoch 525/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0112 - val_loss: 0.0016
Epoch 526/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0109 - val_loss: 0.0016
Epoch 527/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0107 - val_loss: 0.0016
Epoch 528/1000
137/137 [==============================] - 3s 26ms/step - loss: 0.0106 - val_loss: 0.0016
Epoch 529/1000
137/137 [==============================] - 4s 27ms/step - loss: 0.0105 - val_loss: 0.0016
Epoch 530/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0104 - val_loss: 0.0016
Epoch 531/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0103 - val_loss: 0.0016
Epoch 532/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0102 - val_loss: 0.0016
Epoch 533/1000
137/137 [==============================] - 4s 27ms/step - loss: 0.0101 - val_loss: 0.0016
Epoch 534/1000
137/137 [==============================] - 4s 32ms/step - loss: 0.0100 - val_loss: 0.0016
Epoch 535/1000
137/137 [==============================] - 3s 26ms/step - loss: 0.0099 - val_loss: 0.0016
Epoch 536/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0100 - val_loss: 0.0016
Epoch 537/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0103 - val_loss: 0.0016
Epoch 538/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0098 - val_loss: 0.0016
Epoch 539/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0096 - val_loss: 0.0016
Epoch 540/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0095 - val_loss: 0.0016
Epoch 541/1000
137/137 [==============================] - 3s 26ms/step - loss: 0.0094 - val_loss: 0.0016
Epoch 542/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0093 - val_loss: 0.0016
Epoch 543/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0809 - val_loss: 0.0017
Epoch 544/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0115 - val_loss: 0.0017
Epoch 545/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0108 - val_loss: 0.0016
Epoch 546/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0101 - val_loss: 0.0016
Epoch 547/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0099 - val_loss: 0.0016
Epoch 548/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0096 - val_loss: 0.0017
Epoch 549/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0093 - val_loss: 0.0017
Epoch 550/1000
137/137 [==============================] - 4s 27ms/step - loss: 0.0089 - val_loss: 0.0017
Epoch 551/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0087 - val_loss: 0.0017
Epoch 552/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0086 - val_loss: 0.0017
Epoch 553/1000
137/137 [==============================] - 3s 25ms/step - loss: 0.0085 - val_loss: 0.0017
Epoch 554/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0084 - val_loss: 0.0016
Epoch 555/1000
137/137 [==============================] - 4s 31ms/step - loss: 0.0083 - val_loss: 0.0016
Epoch 556/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0082 - val_loss: 0.0017
Epoch 557/1000
137/137 [==============================] - 4s 26ms/step - loss: 0.0082 - val_loss: 0.0017

Actual vs Predicted Yield Spread using vanilla LSTMs

In [ ]:
lag = 50

ts = train['yieldsp'].tail(lag ).values.tolist()

X, _ = deep_learner.create_X_Y(ts, lag=lag )

yhat = model.predict(X)

# to see how our model performs out of sample.
yhat = deep_learner.predict()

fc = train.tail(len(yhat)).copy()
fc['forecast'] = yhat
fc.head()
Out[ ]:
yieldsp forecast
Date
2017-03-03 1.78 1.812001
2017-03-06 1.75 1.769553
2017-03-07 1.76 1.740956
2017-03-08 1.84 1.754811
2017-03-09 1.87 1.845145
In [ ]:
fig_lstm = fc.iplot(asFigure=True, kind='scatter', xTitle='Date', yTitle='Yield Spread')

fig_lstm.update_layout(
    title={
        'text': "Actual and Predicted Yield Spread from vanilla LSTM",
        'y':0.9,
        'x':0.5,
        'xanchor': 'center',
        'yanchor': 'top'})

fig_lstm.show()
In [ ]:
error = mean_squared_error(fc['yieldsp'], fc['forecast'])
rmse = sqrt(error)
print('Train RMSE from vanilla LSTM: %.5f' % rmse)
Train RMSE from vanilla LSTM: 0.04091

Stacked LSTMs with Memory Between Batches

We can gain finer control over when the internal state of the LSTM network is cleared in Keras by making the LSTM layer “stateful”. This means that it can build state over the entire training sequence and even maintain that state if needed to make predictions.

In [ ]:
def create_dataset(dataset, look_back=1):
    X, Y = [], []
    for i in range(len(dataset)-look_back-1):
        a = dataset[i:(i+look_back), 0]
        X.append(a)
        Y.append(dataset[i + look_back, 0])
    return np.array(X), np.array(Y)
In [ ]:
np.random.seed(7)

dataframe = yieldsp
dataset = dataframe.values
dataset = dataset.astype('float32')

# normalize the dataset
scaler = MinMaxScaler(feature_range=(0, 1))
dataset = scaler.fit_transform(dataset)

'''
    Keras requires that In a stateful network, you should only pass inputs
    with a number of samples that can be divided by the batch size.

    Thus, we need to hard code the test-train split, i have done the split in a manner
    that we have 10:1 ratio between train and test sizes.
'''
train, test = dataset[0:8851,:], dataset[8801:-9,:]

# reshape into X=t and Y=t+1
look_back = 50
trainX, trainY = create_dataset(train, look_back)
testX, testY = create_dataset(test, look_back)

# reshape input to be [samples, time steps, features]
trainX = np.reshape(trainX, (trainX.shape[0], trainX.shape[1], 1))
testX = np.reshape(testX, (testX.shape[0], testX.shape[1], 1))
In [ ]:
print(len(trainX))
print(len(testX))
8800
880
In [ ]:
# create and fit the stateful LSTM network

batch_size = 110
model = Sequential()
model.add(LSTM(4, batch_input_shape=(batch_size, look_back, 1), stateful=True, return_sequences=True))
model.add(LSTM(4, batch_input_shape=(batch_size, look_back, 1), stateful=True))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='adam')

model.fit(trainX, trainY, epochs=100, batch_size=batch_size, verbose=1, shuffle=False,
          callbacks = [EarlyStopping(monitor='loss', patience= 20)], )
model.reset_states()

trainPredict = model.predict(trainX, batch_size=batch_size)

model.reset_states()
testPredict = model.predict(testX, batch_size=batch_size)

trainPredict = scaler.inverse_transform(trainPredict)
trainY = scaler.inverse_transform([trainY])
testPredict = scaler.inverse_transform(testPredict)
testY = scaler.inverse_transform([testY])
Epoch 1/100
80/80 [==============================] - 4s 19ms/step - loss: 0.0510
Epoch 2/100
80/80 [==============================] - 2s 19ms/step - loss: 0.0081
Epoch 3/100
80/80 [==============================] - 2s 19ms/step - loss: 0.0032
Epoch 4/100
80/80 [==============================] - 2s 19ms/step - loss: 0.0022
Epoch 5/100
80/80 [==============================] - 2s 20ms/step - loss: 0.0019
Epoch 6/100
80/80 [==============================] - 2s 19ms/step - loss: 0.0017
Epoch 7/100
80/80 [==============================] - 2s 19ms/step - loss: 0.0016
Epoch 8/100
80/80 [==============================] - 2s 19ms/step - loss: 0.0015
Epoch 9/100
80/80 [==============================] - 2s 19ms/step - loss: 0.0014
Epoch 10/100
80/80 [==============================] - 2s 19ms/step - loss: 0.0013
Epoch 11/100
80/80 [==============================] - 2s 19ms/step - loss: 0.0013
Epoch 12/100
80/80 [==============================] - 2s 20ms/step - loss: 0.0012
Epoch 13/100
80/80 [==============================] - 2s 20ms/step - loss: 0.0012
Epoch 14/100
80/80 [==============================] - 2s 19ms/step - loss: 0.0012
Epoch 15/100
80/80 [==============================] - 2s 20ms/step - loss: 0.0011
Epoch 16/100
80/80 [==============================] - 2s 19ms/step - loss: 0.0011
Epoch 17/100
80/80 [==============================] - 2s 19ms/step - loss: 0.0011
Epoch 18/100
80/80 [==============================] - 2s 19ms/step - loss: 0.0010
Epoch 19/100
80/80 [==============================] - 2s 20ms/step - loss: 0.0010
Epoch 20/100
80/80 [==============================] - 2s 20ms/step - loss: 0.0010
Epoch 21/100
80/80 [==============================] - 2s 20ms/step - loss: 9.9109e-04
Epoch 22/100
80/80 [==============================] - 2s 20ms/step - loss: 9.7491e-04
Epoch 23/100
80/80 [==============================] - 2s 22ms/step - loss: 9.5992e-04
Epoch 24/100
80/80 [==============================] - 2s 20ms/step - loss: 9.4591e-04
Epoch 25/100
80/80 [==============================] - 2s 20ms/step - loss: 9.3269e-04
Epoch 26/100
80/80 [==============================] - 2s 20ms/step - loss: 9.2006e-04
Epoch 27/100
80/80 [==============================] - 2s 20ms/step - loss: 9.0786e-04
Epoch 28/100
80/80 [==============================] - 2s 20ms/step - loss: 8.9596e-04
Epoch 29/100
80/80 [==============================] - 2s 20ms/step - loss: 8.8427e-04
Epoch 30/100
80/80 [==============================] - 2s 20ms/step - loss: 8.7275e-04
Epoch 31/100
80/80 [==============================] - 2s 20ms/step - loss: 8.6137e-04
Epoch 32/100
80/80 [==============================] - 2s 20ms/step - loss: 8.5014e-04
Epoch 33/100
80/80 [==============================] - 2s 20ms/step - loss: 8.3903e-04
Epoch 34/100
80/80 [==============================] - 2s 20ms/step - loss: 8.2806e-04
Epoch 35/100
80/80 [==============================] - 2s 20ms/step - loss: 8.1722e-04
Epoch 36/100
80/80 [==============================] - 2s 20ms/step - loss: 8.0652e-04
Epoch 37/100
80/80 [==============================] - 2s 20ms/step - loss: 7.9593e-04
Epoch 38/100
80/80 [==============================] - 2s 20ms/step - loss: 7.8545e-04
Epoch 39/100
80/80 [==============================] - 2s 20ms/step - loss: 7.7508e-04
Epoch 40/100
80/80 [==============================] - 2s 20ms/step - loss: 7.6480e-04
Epoch 41/100
80/80 [==============================] - 2s 20ms/step - loss: 7.5460e-04
Epoch 42/100
80/80 [==============================] - 2s 20ms/step - loss: 7.4447e-04
Epoch 43/100
80/80 [==============================] - 2s 19ms/step - loss: 7.3441e-04
Epoch 44/100
80/80 [==============================] - 2s 20ms/step - loss: 7.2440e-04
Epoch 45/100
80/80 [==============================] - 2s 20ms/step - loss: 7.1443e-04
Epoch 46/100
80/80 [==============================] - 2s 20ms/step - loss: 7.0451e-04
Epoch 47/100
80/80 [==============================] - 2s 29ms/step - loss: 6.9463e-04
Epoch 48/100
80/80 [==============================] - 2s 22ms/step - loss: 6.8478e-04
Epoch 49/100
80/80 [==============================] - 2s 21ms/step - loss: 6.7497e-04
Epoch 50/100
80/80 [==============================] - 2s 21ms/step - loss: 6.6519e-04
Epoch 51/100
80/80 [==============================] - 2s 20ms/step - loss: 6.5544e-04
Epoch 52/100
80/80 [==============================] - 2s 21ms/step - loss: 6.4573e-04
Epoch 53/100
80/80 [==============================] - 2s 20ms/step - loss: 6.3606e-04
Epoch 54/100
80/80 [==============================] - 2s 20ms/step - loss: 6.2643e-04
Epoch 55/100
80/80 [==============================] - 2s 21ms/step - loss: 6.1684e-04
Epoch 56/100
80/80 [==============================] - 2s 20ms/step - loss: 6.0731e-04
Epoch 57/100
80/80 [==============================] - 2s 20ms/step - loss: 5.9784e-04
Epoch 58/100
80/80 [==============================] - 2s 20ms/step - loss: 5.8844e-04
Epoch 59/100
80/80 [==============================] - 2s 20ms/step - loss: 5.7910e-04
Epoch 60/100
80/80 [==============================] - 2s 21ms/step - loss: 5.6985e-04
Epoch 61/100
80/80 [==============================] - 2s 20ms/step - loss: 5.6069e-04
Epoch 62/100
80/80 [==============================] - 2s 20ms/step - loss: 5.5163e-04
Epoch 63/100
80/80 [==============================] - 2s 20ms/step - loss: 5.4267e-04
Epoch 64/100
80/80 [==============================] - 2s 20ms/step - loss: 5.3382e-04
Epoch 65/100
80/80 [==============================] - 2s 20ms/step - loss: 5.2510e-04
Epoch 66/100
80/80 [==============================] - 2s 20ms/step - loss: 5.1650e-04
Epoch 67/100
80/80 [==============================] - 2s 20ms/step - loss: 5.0804e-04
Epoch 68/100
80/80 [==============================] - 2s 20ms/step - loss: 4.9971e-04
Epoch 69/100
80/80 [==============================] - 2s 20ms/step - loss: 4.9153e-04
Epoch 70/100
80/80 [==============================] - 2s 20ms/step - loss: 4.8349e-04
Epoch 71/100
80/80 [==============================] - 2s 20ms/step - loss: 4.7560e-04
Epoch 72/100
80/80 [==============================] - 2s 20ms/step - loss: 4.6786e-04
Epoch 73/100
80/80 [==============================] - 2s 20ms/step - loss: 4.6027e-04
Epoch 74/100
80/80 [==============================] - 2s 20ms/step - loss: 4.5283e-04
Epoch 75/100
80/80 [==============================] - 2s 20ms/step - loss: 4.4554e-04
Epoch 76/100
80/80 [==============================] - 2s 20ms/step - loss: 4.3840e-04
Epoch 77/100
80/80 [==============================] - 2s 20ms/step - loss: 4.3141e-04
Epoch 78/100
80/80 [==============================] - 2s 20ms/step - loss: 4.2456e-04
Epoch 79/100
80/80 [==============================] - 2s 19ms/step - loss: 4.1784e-04
Epoch 80/100
80/80 [==============================] - 2s 19ms/step - loss: 4.1126e-04
Epoch 81/100
80/80 [==============================] - 2s 20ms/step - loss: 4.0480e-04
Epoch 82/100
80/80 [==============================] - 2s 20ms/step - loss: 3.9847e-04
Epoch 83/100
80/80 [==============================] - 2s 19ms/step - loss: 3.9226e-04
Epoch 84/100
80/80 [==============================] - 2s 20ms/step - loss: 3.8615e-04
Epoch 85/100
80/80 [==============================] - 2s 19ms/step - loss: 3.8015e-04
Epoch 86/100
80/80 [==============================] - 2s 28ms/step - loss: 3.7426e-04
Epoch 87/100
80/80 [==============================] - 2s 22ms/step - loss: 3.6847e-04
Epoch 88/100
80/80 [==============================] - 2s 21ms/step - loss: 3.6278e-04
Epoch 89/100
80/80 [==============================] - 2s 21ms/step - loss: 3.5719e-04
Epoch 90/100
80/80 [==============================] - 2s 20ms/step - loss: 3.5171e-04
Epoch 91/100
80/80 [==============================] - 2s 21ms/step - loss: 3.4635e-04
Epoch 92/100
80/80 [==============================] - 2s 19ms/step - loss: 3.4113e-04
Epoch 93/100
80/80 [==============================] - 2s 20ms/step - loss: 3.3605e-04
Epoch 94/100
80/80 [==============================] - 2s 20ms/step - loss: 3.3115e-04
Epoch 95/100
80/80 [==============================] - 2s 21ms/step - loss: 3.2643e-04
Epoch 96/100
80/80 [==============================] - 2s 20ms/step - loss: 3.2192e-04
Epoch 97/100
80/80 [==============================] - 2s 20ms/step - loss: 3.1764e-04
Epoch 98/100
80/80 [==============================] - 2s 20ms/step - loss: 3.1358e-04
Epoch 99/100
80/80 [==============================] - 2s 22ms/step - loss: 3.0976e-04
Epoch 100/100
80/80 [==============================] - 2s 20ms/step - loss: 3.0616e-04

Results from using Stateful stacked LSTMs

In [ ]:
# calculate root mean squared error
trainScore = math.sqrt(mean_squared_error(trainY[0], trainPredict[:,0]))
print('Train Score: %.5f RMSE' % (trainScore))
testScore = math.sqrt(mean_squared_error(testY[0], testPredict[:,0]))
print('Test Score: %.5f RMSE' % (testScore))
Train Score: 0.08988 RMSE
Test Score: 0.06424 RMSE
In [ ]:
# shift train predictions for plotting
trainPredictPlot = np.empty_like(dataset)
trainPredictPlot[:, :] = np.nan
trainPredictPlot[look_back:len(trainPredict)+look_back, :] = trainPredict

# shift test predictions for plotting
testPredictPlot = np.empty_like(dataset)
testPredictPlot[:, :] = np.nan
testPredictPlot[len(trainPredict) + look_back:9730, :] = testPredict

# combine the predicted and actual data
train_pred = pd.DataFrame(trainPredictPlot, columns = ['train_pred'])
test_pred = pd.DataFrame(testPredictPlot, columns = ['test_pred'])
pred = pd.merge(train_pred, test_pred, on = test_pred.index)
pred1 = pred[['train_pred', 'test_pred']]
merged = pd.merge(yieldsp, pred1, on = yieldsp.index).rename(columns= {'key_0': 'Date'}).set_index('Date')

fig_lstm = merged.iplot(asFigure=True, kind='scatter', xTitle='Date', yTitle='Yield Spread')
fig_lstm

LSTM using multivariate data

Building a multivariate LSTM model

In [ ]:
def series_to_supervised(data, n_in=1, n_out=1, dropnan=True):
	n_vars = 1 if type(data) is list else data.shape[1]
	df = DataFrame(data)
	cols, names = list(), list()
 
	for i in range(n_in, 0, -1):
		cols.append(df.shift(i))
		names += [('var%d(t-%d)' % (j+1, i)) for j in range(n_vars)]
    
	for i in range(0, n_out):
		cols.append(df.shift(-i))
		if i == 0:
			names += [('var%d(t)' % (j+1)) for j in range(n_vars)]
		else:
			names += [('var%d(t+%d)' % (j+1, i)) for j in range(n_vars)]
    
	agg = pd.concat(cols, axis=1)
	agg.columns = names
    
	if dropnan:
		agg.dropna(inplace=True)
	return agg
 
values = yield_d.values

encoder = LabelEncoder()
values[:,4] = encoder.fit_transform(values[:,4])
values = values.astype('float32')

scaler = MinMaxScaler(feature_range=(0, 1))
scaled = scaler.fit_transform(values)

reframed = series_to_supervised(scaled, 1, 1)
reframed.drop(reframed.columns[[ 10, 11, 12,13, 14, 15, 16, 17]], axis=1, inplace=True)
reframed.head()
Out[ ]:
var1(t-1) var2(t-1) var3(t-1) var4(t-1) var5(t-1) var6(t-1) var7(t-1) var8(t-1) var9(t-1) var1(t)
1 0.219917 0.158129 0.867505 0.0 0.257310 0.713155 0.043947 0.774314 0.117786 0.219917
2 0.219917 0.158129 0.867505 0.0 0.257310 0.713155 0.044159 0.778583 0.117786 0.217842
3 0.217842 0.146993 0.872790 0.0 0.318713 0.719705 0.044351 0.782474 0.129367 0.226141
4 0.226141 0.155902 0.871609 0.0 0.333333 0.720388 0.044525 0.785997 0.141925 0.238589
5 0.238589 0.167038 0.870737 0.0 0.350877 0.723854 0.044680 0.789162 0.152775 0.244813
In [ ]:
values = reframed.values
n_train_hours = int(.95 *len(reframed))
train = values[:n_train_hours, :]
test = values[n_train_hours:, :]

train_X, train_y = train[:, :-1], train[:, -1]
test_X, test_y = test[:, :-1], test[:, -1]

train_X = train_X.reshape((train_X.shape[0], 1, train_X.shape[1]))
test_X = test_X.reshape((test_X.shape[0], 1, test_X.shape[1]))
print(train_X.shape, train_y.shape, test_X.shape, test_y.shape)
(10669, 1, 9) (10669,) (562, 1, 9) (562,)
In [ ]:
model = Sequential()
model.add(LSTM(25, input_shape=(train_X.shape[1], train_X.shape[2])))
model.add(Dense(1))
model.compile(loss='mae', optimizer='RMSprop')

# fit network
history = model.fit(train_X, train_y, epochs=500, batch_size=50, validation_data=(test_X, test_y), 
                    # callbacks = [EarlyStopping(monitor='val_loss', patience= 100)],
                    verbose=1, shuffle=False)
Epoch 1/500
214/214 [==============================] - 2s 4ms/step - loss: 0.3040 - val_loss: 0.1728
Epoch 2/500
214/214 [==============================] - 0s 2ms/step - loss: 0.1179 - val_loss: 0.1278
Epoch 3/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0622 - val_loss: 0.1142
Epoch 4/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0342 - val_loss: 0.0833
Epoch 5/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0221 - val_loss: 0.0555
Epoch 6/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0201 - val_loss: 0.0367
Epoch 7/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0180 - val_loss: 0.0264
Epoch 8/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0156 - val_loss: 0.0201
Epoch 9/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0147 - val_loss: 0.0131
Epoch 10/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0147 - val_loss: 0.0131
Epoch 11/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0139 - val_loss: 0.0176
Epoch 12/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0138 - val_loss: 0.0227
Epoch 13/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0137 - val_loss: 0.0263
Epoch 14/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0136 - val_loss: 0.0286
Epoch 15/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0137 - val_loss: 0.0290
Epoch 16/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0136 - val_loss: 0.0334
Epoch 17/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0135 - val_loss: 0.0340
Epoch 18/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0134 - val_loss: 0.0344
Epoch 19/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0131 - val_loss: 0.0357
Epoch 20/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0132 - val_loss: 0.0356
Epoch 21/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0130 - val_loss: 0.0371
Epoch 22/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0128 - val_loss: 0.0370
Epoch 23/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0125 - val_loss: 0.0382
Epoch 24/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0126 - val_loss: 0.0383
Epoch 25/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0125 - val_loss: 0.0386
Epoch 26/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0121 - val_loss: 0.0392
Epoch 27/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0120 - val_loss: 0.0398
Epoch 28/500
214/214 [==============================] - 1s 3ms/step - loss: 0.0122 - val_loss: 0.0387
Epoch 29/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0120 - val_loss: 0.0383
Epoch 30/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0115 - val_loss: 0.0378
Epoch 31/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0120 - val_loss: 0.0390
Epoch 32/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0116 - val_loss: 0.0394
Epoch 33/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0118 - val_loss: 0.0393
Epoch 34/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0115 - val_loss: 0.0395
Epoch 35/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0117 - val_loss: 0.0389
Epoch 36/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0113 - val_loss: 0.0391
Epoch 37/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0113 - val_loss: 0.0400
Epoch 38/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0113 - val_loss: 0.0398
Epoch 39/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0112 - val_loss: 0.0395
Epoch 40/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0111 - val_loss: 0.0400
Epoch 41/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0111 - val_loss: 0.0400
Epoch 42/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0110 - val_loss: 0.0404
Epoch 43/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0110 - val_loss: 0.0402
Epoch 44/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0109 - val_loss: 0.0407
Epoch 45/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0108 - val_loss: 0.0410
Epoch 46/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0109 - val_loss: 0.0402
Epoch 47/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0107 - val_loss: 0.0415
Epoch 48/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0108 - val_loss: 0.0402
Epoch 49/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0106 - val_loss: 0.0399
Epoch 50/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0106 - val_loss: 0.0412
Epoch 51/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0105 - val_loss: 0.0402
Epoch 52/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0105 - val_loss: 0.0402
Epoch 53/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0104 - val_loss: 0.0401
Epoch 54/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0104 - val_loss: 0.0400
Epoch 55/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0104 - val_loss: 0.0405
Epoch 56/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0104 - val_loss: 0.0402
Epoch 57/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0104 - val_loss: 0.0390
Epoch 58/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0103 - val_loss: 0.0394
Epoch 59/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0104 - val_loss: 0.0398
Epoch 60/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0103 - val_loss: 0.0402
Epoch 61/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0103 - val_loss: 0.0402
Epoch 62/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0103 - val_loss: 0.0403
Epoch 63/500
214/214 [==============================] - 1s 3ms/step - loss: 0.0101 - val_loss: 0.0391
Epoch 64/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0103 - val_loss: 0.0381
Epoch 65/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0101 - val_loss: 0.0374
Epoch 66/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0102 - val_loss: 0.0367
Epoch 67/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0102 - val_loss: 0.0364
Epoch 68/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0101 - val_loss: 0.0368
Epoch 69/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0102 - val_loss: 0.0365
Epoch 70/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0102 - val_loss: 0.0357
Epoch 71/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0101 - val_loss: 0.0357
Epoch 72/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0100 - val_loss: 0.0343
Epoch 73/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0099 - val_loss: 0.0326
Epoch 74/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0099 - val_loss: 0.0337
Epoch 75/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0100 - val_loss: 0.0346
Epoch 76/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0100 - val_loss: 0.0345
Epoch 77/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0099 - val_loss: 0.0337
Epoch 78/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0099 - val_loss: 0.0339
Epoch 79/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0098 - val_loss: 0.0334
Epoch 80/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0095 - val_loss: 0.0326
Epoch 81/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0096 - val_loss: 0.0327
Epoch 82/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0095 - val_loss: 0.0339
Epoch 83/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0097 - val_loss: 0.0333
Epoch 84/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0096 - val_loss: 0.0337
Epoch 85/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0095 - val_loss: 0.0343
Epoch 86/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0095 - val_loss: 0.0343
Epoch 87/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0095 - val_loss: 0.0341
Epoch 88/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0095 - val_loss: 0.0337
Epoch 89/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0093 - val_loss: 0.0330
Epoch 90/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0094 - val_loss: 0.0326
Epoch 91/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0096 - val_loss: 0.0334
Epoch 92/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0094 - val_loss: 0.0332
Epoch 93/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0096 - val_loss: 0.0339
Epoch 94/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0094 - val_loss: 0.0343
Epoch 95/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0094 - val_loss: 0.0341
Epoch 96/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0096 - val_loss: 0.0350
Epoch 97/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0095 - val_loss: 0.0369
Epoch 98/500
214/214 [==============================] - 1s 3ms/step - loss: 0.0094 - val_loss: 0.0344
Epoch 99/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0095 - val_loss: 0.0356
Epoch 100/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0093 - val_loss: 0.0337
Epoch 101/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0095 - val_loss: 0.0350
Epoch 102/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0093 - val_loss: 0.0329
Epoch 103/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0094 - val_loss: 0.0338
Epoch 104/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0091 - val_loss: 0.0320
Epoch 105/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0091 - val_loss: 0.0313
Epoch 106/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0095 - val_loss: 0.0322
Epoch 107/500
214/214 [==============================] - 1s 5ms/step - loss: 0.0091 - val_loss: 0.0299
Epoch 108/500
214/214 [==============================] - 1s 3ms/step - loss: 0.0094 - val_loss: 0.0316
Epoch 109/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0090 - val_loss: 0.0321
Epoch 110/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0091 - val_loss: 0.0307
Epoch 111/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0095 - val_loss: 0.0314
Epoch 112/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0091 - val_loss: 0.0308
Epoch 113/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0094 - val_loss: 0.0315
Epoch 114/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0090 - val_loss: 0.0330
Epoch 115/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0092 - val_loss: 0.0312
Epoch 116/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0090 - val_loss: 0.0316
Epoch 117/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0090 - val_loss: 0.0304
Epoch 118/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0093 - val_loss: 0.0313
Epoch 119/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0091 - val_loss: 0.0304
Epoch 120/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0093 - val_loss: 0.0323
Epoch 121/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0089 - val_loss: 0.0314
Epoch 122/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0088 - val_loss: 0.0305
Epoch 123/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0093 - val_loss: 0.0312
Epoch 124/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0090 - val_loss: 0.0305
Epoch 125/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0093 - val_loss: 0.0307
Epoch 126/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0092 - val_loss: 0.0314
Epoch 127/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0091 - val_loss: 0.0314
Epoch 128/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0091 - val_loss: 0.0306
Epoch 129/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0093 - val_loss: 0.0317
Epoch 130/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0091 - val_loss: 0.0309
Epoch 131/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0089 - val_loss: 0.0301
Epoch 132/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0090 - val_loss: 0.0297
Epoch 133/500
214/214 [==============================] - 1s 3ms/step - loss: 0.0089 - val_loss: 0.0296
Epoch 134/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0089 - val_loss: 0.0294
Epoch 135/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0089 - val_loss: 0.0298
Epoch 136/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0089 - val_loss: 0.0301
Epoch 137/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0089 - val_loss: 0.0301
Epoch 138/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0089 - val_loss: 0.0301
Epoch 139/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0090 - val_loss: 0.0315
Epoch 140/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0091 - val_loss: 0.0308
Epoch 141/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0091 - val_loss: 0.0307
Epoch 142/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0090 - val_loss: 0.0319
Epoch 143/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0091 - val_loss: 0.0301
Epoch 144/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0093 - val_loss: 0.0325
Epoch 145/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0091 - val_loss: 0.0310
Epoch 146/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0091 - val_loss: 0.0326
Epoch 147/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0090 - val_loss: 0.0315
Epoch 148/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0090 - val_loss: 0.0326
Epoch 149/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0091 - val_loss: 0.0306
Epoch 150/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0091 - val_loss: 0.0308
Epoch 151/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0090 - val_loss: 0.0317
Epoch 152/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0090 - val_loss: 0.0308
Epoch 153/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0089 - val_loss: 0.0320
Epoch 154/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0089 - val_loss: 0.0306
Epoch 155/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0088 - val_loss: 0.0299
Epoch 156/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0089 - val_loss: 0.0308
Epoch 157/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0089 - val_loss: 0.0300
Epoch 158/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0089 - val_loss: 0.0297
Epoch 159/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0090 - val_loss: 0.0312
Epoch 160/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0088 - val_loss: 0.0302
Epoch 161/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0089 - val_loss: 0.0293
Epoch 162/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0089 - val_loss: 0.0300
Epoch 163/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0088 - val_loss: 0.0305
Epoch 164/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0088 - val_loss: 0.0290
Epoch 165/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0089 - val_loss: 0.0289
Epoch 166/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0088 - val_loss: 0.0286
Epoch 167/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0088 - val_loss: 0.0278
Epoch 168/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0087 - val_loss: 0.0278
Epoch 169/500
214/214 [==============================] - 1s 3ms/step - loss: 0.0088 - val_loss: 0.0277
Epoch 170/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0088 - val_loss: 0.0272
Epoch 171/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0087 - val_loss: 0.0268
Epoch 172/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0088 - val_loss: 0.0265
Epoch 173/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0086 - val_loss: 0.0265
Epoch 174/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0087 - val_loss: 0.0272
Epoch 175/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0088 - val_loss: 0.0275
Epoch 176/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0087 - val_loss: 0.0273
Epoch 177/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0088 - val_loss: 0.0258
Epoch 178/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0089 - val_loss: 0.0276
Epoch 179/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0087 - val_loss: 0.0276
Epoch 180/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0087 - val_loss: 0.0264
Epoch 181/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0087 - val_loss: 0.0267
Epoch 182/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0088 - val_loss: 0.0267
Epoch 183/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0089 - val_loss: 0.0280
Epoch 184/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0089 - val_loss: 0.0276
Epoch 185/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0089 - val_loss: 0.0277
Epoch 186/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0088 - val_loss: 0.0277
Epoch 187/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0090 - val_loss: 0.0267
Epoch 188/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0086 - val_loss: 0.0259
Epoch 189/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0087 - val_loss: 0.0275
Epoch 190/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0087 - val_loss: 0.0270
Epoch 191/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0087 - val_loss: 0.0260
Epoch 192/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0086 - val_loss: 0.0265
Epoch 193/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0086 - val_loss: 0.0260
Epoch 194/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0086 - val_loss: 0.0273
Epoch 195/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0086 - val_loss: 0.0262
Epoch 196/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0088 - val_loss: 0.0275
Epoch 197/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0268
Epoch 198/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0256
Epoch 199/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0086 - val_loss: 0.0258
Epoch 200/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0087 - val_loss: 0.0257
Epoch 201/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0086 - val_loss: 0.0254
Epoch 202/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0088 - val_loss: 0.0262
Epoch 203/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0087 - val_loss: 0.0260
Epoch 204/500
214/214 [==============================] - 1s 3ms/step - loss: 0.0085 - val_loss: 0.0258
Epoch 205/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0088 - val_loss: 0.0272
Epoch 206/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0087 - val_loss: 0.0266
Epoch 207/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0270
Epoch 208/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0256
Epoch 209/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0257
Epoch 210/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0088 - val_loss: 0.0264
Epoch 211/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0257
Epoch 212/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0087 - val_loss: 0.0262
Epoch 213/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0087 - val_loss: 0.0262
Epoch 214/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0088 - val_loss: 0.0264
Epoch 215/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0251
Epoch 216/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0086 - val_loss: 0.0260
Epoch 217/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0087 - val_loss: 0.0262
Epoch 218/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0086 - val_loss: 0.0263
Epoch 219/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0087 - val_loss: 0.0263
Epoch 220/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0087 - val_loss: 0.0276
Epoch 221/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0086 - val_loss: 0.0261
Epoch 222/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0086 - val_loss: 0.0260
Epoch 223/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0087 - val_loss: 0.0263
Epoch 224/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0251
Epoch 225/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0087 - val_loss: 0.0267
Epoch 226/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0255
Epoch 227/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0086 - val_loss: 0.0247
Epoch 228/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0249
Epoch 229/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0252
Epoch 230/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0258
Epoch 231/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0253
Epoch 232/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0087 - val_loss: 0.0254
Epoch 233/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0241
Epoch 234/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0086 - val_loss: 0.0248
Epoch 235/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0257
Epoch 236/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0087 - val_loss: 0.0255
Epoch 237/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0255
Epoch 238/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0247
Epoch 239/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0254
Epoch 240/500
214/214 [==============================] - 1s 3ms/step - loss: 0.0085 - val_loss: 0.0244
Epoch 241/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0086 - val_loss: 0.0251
Epoch 242/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0242
Epoch 243/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0242
Epoch 244/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0241
Epoch 245/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0243
Epoch 246/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0086 - val_loss: 0.0237
Epoch 247/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0243
Epoch 248/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0229
Epoch 249/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0086 - val_loss: 0.0231
Epoch 250/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0243
Epoch 251/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0227
Epoch 252/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0235
Epoch 253/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0229
Epoch 254/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0086 - val_loss: 0.0236
Epoch 255/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0239
Epoch 256/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0225
Epoch 257/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0243
Epoch 258/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0239
Epoch 259/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0231
Epoch 260/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0247
Epoch 261/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0240
Epoch 262/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0226
Epoch 263/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0243
Epoch 264/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0234
Epoch 265/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0225
Epoch 266/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0232
Epoch 267/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0229
Epoch 268/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0215
Epoch 269/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0221
Epoch 270/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0223
Epoch 271/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0232
Epoch 272/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0227
Epoch 273/500
214/214 [==============================] - 1s 5ms/step - loss: 0.0083 - val_loss: 0.0217
Epoch 274/500
214/214 [==============================] - 1s 2ms/step - loss: 0.0085 - val_loss: 0.0224
Epoch 275/500
214/214 [==============================] - 1s 4ms/step - loss: 0.0086 - val_loss: 0.0234
Epoch 276/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0233
Epoch 277/500
214/214 [==============================] - 1s 2ms/step - loss: 0.0085 - val_loss: 0.0225
Epoch 278/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0238
Epoch 279/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0229
Epoch 280/500
214/214 [==============================] - 1s 2ms/step - loss: 0.0083 - val_loss: 0.0223
Epoch 281/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0213
Epoch 282/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0220
Epoch 283/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0214
Epoch 284/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0231
Epoch 285/500
214/214 [==============================] - 1s 2ms/step - loss: 0.0083 - val_loss: 0.0227
Epoch 286/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0225
Epoch 287/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0231
Epoch 288/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0237
Epoch 289/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0231
Epoch 290/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0233
Epoch 291/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0234
Epoch 292/500
214/214 [==============================] - 1s 3ms/step - loss: 0.0082 - val_loss: 0.0224
Epoch 293/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0233
Epoch 294/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0229
Epoch 295/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0082 - val_loss: 0.0226
Epoch 296/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0229
Epoch 297/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0225
Epoch 298/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0227
Epoch 299/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0082 - val_loss: 0.0223
Epoch 300/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0230
Epoch 301/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0224
Epoch 302/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0219
Epoch 303/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0216
Epoch 304/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0215
Epoch 305/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0214
Epoch 306/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0207
Epoch 307/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0213
Epoch 308/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0208
Epoch 309/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0203
Epoch 310/500
214/214 [==============================] - 1s 3ms/step - loss: 0.0085 - val_loss: 0.0213
Epoch 311/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0211
Epoch 312/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0206
Epoch 313/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0219
Epoch 314/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0206
Epoch 315/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0220
Epoch 316/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0203
Epoch 317/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0205
Epoch 318/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0215
Epoch 319/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0204
Epoch 320/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0193
Epoch 321/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0202
Epoch 322/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0194
Epoch 323/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0207
Epoch 324/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0201
Epoch 325/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0202
Epoch 326/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0204
Epoch 327/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0082 - val_loss: 0.0200
Epoch 328/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0215
Epoch 329/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0211
Epoch 330/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0197
Epoch 331/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0208
Epoch 332/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0205
Epoch 333/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0205
Epoch 334/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0203
Epoch 335/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0193
Epoch 336/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0184
Epoch 337/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0206
Epoch 338/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0195
Epoch 339/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0082 - val_loss: 0.0180
Epoch 340/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0201
Epoch 341/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0195
Epoch 342/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0184
Epoch 343/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0192
Epoch 344/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0204
Epoch 345/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0189
Epoch 346/500
214/214 [==============================] - 1s 3ms/step - loss: 0.0083 - val_loss: 0.0205
Epoch 347/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0190
Epoch 348/500
214/214 [==============================] - 1s 2ms/step - loss: 0.0084 - val_loss: 0.0192
Epoch 349/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0202
Epoch 350/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0183
Epoch 351/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0201
Epoch 352/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0185
Epoch 353/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0193
Epoch 354/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0188
Epoch 355/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0192
Epoch 356/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0082 - val_loss: 0.0186
Epoch 357/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0189
Epoch 358/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0195
Epoch 359/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0187
Epoch 360/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0186
Epoch 361/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0190
Epoch 362/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0198
Epoch 363/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0185
Epoch 364/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0192
Epoch 365/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0196
Epoch 366/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0186
Epoch 367/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0082 - val_loss: 0.0191
Epoch 368/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0195
Epoch 369/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0082 - val_loss: 0.0187
Epoch 370/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0196
Epoch 371/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0202
Epoch 372/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0197
Epoch 373/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0203
Epoch 374/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0082 - val_loss: 0.0183
Epoch 375/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0082 - val_loss: 0.0189
Epoch 376/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0196
Epoch 377/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0082 - val_loss: 0.0199
Epoch 378/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0082 - val_loss: 0.0185
Epoch 379/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0196
Epoch 380/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0082 - val_loss: 0.0185
Epoch 381/500
214/214 [==============================] - 1s 3ms/step - loss: 0.0083 - val_loss: 0.0190
Epoch 382/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0196
Epoch 383/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0192
Epoch 384/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0196
Epoch 385/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0193
Epoch 386/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0197
Epoch 387/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0187
Epoch 388/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0082 - val_loss: 0.0200
Epoch 389/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0186
Epoch 390/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0194
Epoch 391/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0191
Epoch 392/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0182
Epoch 393/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0184
Epoch 394/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0082 - val_loss: 0.0192
Epoch 395/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0082 - val_loss: 0.0178
Epoch 396/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0182
Epoch 397/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0186
Epoch 398/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0188
Epoch 399/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0082 - val_loss: 0.0174
Epoch 400/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0166
Epoch 401/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0163
Epoch 402/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0160
Epoch 403/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0160
Epoch 404/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0157
Epoch 405/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0169
Epoch 406/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0159
Epoch 407/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0165
Epoch 408/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0156
Epoch 409/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0161
Epoch 410/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0155
Epoch 411/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0167
Epoch 412/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0160
Epoch 413/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0155
Epoch 414/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0159
Epoch 415/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0162
Epoch 416/500
214/214 [==============================] - 1s 3ms/step - loss: 0.0082 - val_loss: 0.0171
Epoch 417/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0157
Epoch 418/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0157
Epoch 419/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0157
Epoch 420/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0154
Epoch 421/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0171
Epoch 422/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0155
Epoch 423/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0082 - val_loss: 0.0162
Epoch 424/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0151
Epoch 425/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0160
Epoch 426/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0149
Epoch 427/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0154
Epoch 428/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0154
Epoch 429/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0160
Epoch 430/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0153
Epoch 431/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0160
Epoch 432/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0154
Epoch 433/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0147
Epoch 434/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0082 - val_loss: 0.0157
Epoch 435/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0146
Epoch 436/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0139
Epoch 437/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0138
Epoch 438/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0082 - val_loss: 0.0153
Epoch 439/500
214/214 [==============================] - 1s 4ms/step - loss: 0.0083 - val_loss: 0.0149
Epoch 440/500
214/214 [==============================] - 1s 3ms/step - loss: 0.0083 - val_loss: 0.0139
Epoch 441/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0136
Epoch 442/500
214/214 [==============================] - 1s 2ms/step - loss: 0.0083 - val_loss: 0.0149
Epoch 443/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0138
Epoch 444/500
214/214 [==============================] - 1s 2ms/step - loss: 0.0084 - val_loss: 0.0137
Epoch 445/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0133
Epoch 446/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0133
Epoch 447/500
214/214 [==============================] - 1s 2ms/step - loss: 0.0083 - val_loss: 0.0134
Epoch 448/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0135
Epoch 449/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0131
Epoch 450/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0129
Epoch 451/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0129
Epoch 452/500
214/214 [==============================] - 1s 4ms/step - loss: 0.0084 - val_loss: 0.0131
Epoch 453/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0132
Epoch 454/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0129
Epoch 455/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0085 - val_loss: 0.0133
Epoch 456/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0127
Epoch 457/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0125
Epoch 458/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0082 - val_loss: 0.0127
Epoch 459/500
214/214 [==============================] - 1s 2ms/step - loss: 0.0083 - val_loss: 0.0126
Epoch 460/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0128
Epoch 461/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0125
Epoch 462/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0130
Epoch 463/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0125
Epoch 464/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0120
Epoch 465/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0082 - val_loss: 0.0135
Epoch 466/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0130
Epoch 467/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0123
Epoch 468/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0135
Epoch 469/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0127
Epoch 470/500
214/214 [==============================] - 1s 2ms/step - loss: 0.0084 - val_loss: 0.0125
Epoch 471/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0121
Epoch 472/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0123
Epoch 473/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0121
Epoch 474/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0118
Epoch 475/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0129
Epoch 476/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0119
Epoch 477/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0118
Epoch 478/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0118
Epoch 479/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0131
Epoch 480/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0121
Epoch 481/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0120
Epoch 482/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0130
Epoch 483/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0120
Epoch 484/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0119
Epoch 485/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0118
Epoch 486/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0126
Epoch 487/500
214/214 [==============================] - 1s 4ms/step - loss: 0.0083 - val_loss: 0.0115
Epoch 488/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0115
Epoch 489/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0113
Epoch 490/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0111
Epoch 491/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0083 - val_loss: 0.0110
Epoch 492/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0108
Epoch 493/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0103
Epoch 494/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0102
Epoch 495/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0123
Epoch 496/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0116
Epoch 497/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0113
Epoch 498/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0107
Epoch 499/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0124
Epoch 500/500
214/214 [==============================] - 0s 2ms/step - loss: 0.0084 - val_loss: 0.0114

Plotting the test and validation losses

In [ ]:
mod_loss1 = pd.DataFrame(history.history, columns=['loss', 'val_loss'])
In [ ]:
fig_loss1 = mod_loss1.iplot(asFigure=True, kind='scatter', xTitle='Epochs', yTitle='Loss')

fig_loss1.update_layout(
    title={
        'text': 'Training and Validation loss of the multivariate LSTM model ',
        'y':0.9,
        'x':0.5,
        'xanchor': 'center',
        'yanchor': 'top'})

fig_loss1.show()

Forecasting using the multivariate LSTM model

In [ ]:
# forecast for the entire dataset
yhat = model.predict(test_X)
test_X = test_X.reshape((test_X.shape[0], test_X.shape[2]))

# invert scaling for forecast
inv_yhat = concatenate((yhat, test_X[:, 1:]), axis=1)
inv_yhat = scaler.inverse_transform(inv_yhat)
inv_yhat = inv_yhat[:,0]

# invert scaling for actual
test_y = test_y.reshape((len(test_y), 1))
inv_y = concatenate((test_y, test_X[:, 1:]), axis=1)
inv_y = scaler.inverse_transform(inv_y)
inv_y = inv_y[:,0]

# calculate RMSE
rmse = sqrt(mean_squared_error(inv_y, inv_yhat))
print('Test RMSE: %.5f' % rmse)
Test RMSE: 0.07208
In [ ]:
train_data, test_data = np.split(yield_d, [int(.95 *len(yield_d))])

invert = pd.DataFrame(inv_y, columns = ['yieldsp_lstm'])
yhat_lstm = pd.DataFrame(inv_yhat, columns = ['inv_yhat_lstm'])
lstm = pd.merge(invert, yhat_lstm, 
                on = test_data.index).rename(columns= {'key_0': 'Date'}).set_index('Date')
In [ ]:
fig_lstm_d = lstm.iplot(asFigure=True, kind='scatter', xTitle='Date', yTitle='Yield Spread')
fig_lstm_d.update_layout(
    title={
        'text': 'Actual and Forecasted Yield Spread using Multivariate LSTM model',
        'y':0.9,
        'x':0.5,
        'xanchor': 'center',
        'yanchor': 'top'})

fig_lstm_d.show()

Conclusion

Model RMSE
ARIMA(1,0,3) 0.05110
Differenced ARIMA(3,1,3) 0.05147
SARIMAX(2,1,2) 0.4744
VAR(45) 0.50267
MLP 0.055239
Stateful stacked LSTM 0.06081
Multivariate LSTM 0.07118

The results presented here are from the replication-minimal notebook and differ slightly to the ones obtained in this notebook due to the randomness in training Neural nets.